WO1997010565A1

WO1997010565A1 - Indicia device and optical scanning method

Info

Publication number: WO1997010565A1
Application number: PCT/FR1996/001389
Authority: WO
Inventors: Jean-Pierre Moussette
Original assignee: Moussette Jean Pierre
Priority date: 1995-09-13
Filing date: 1996-09-09
Publication date: 1997-03-20
Also published as: FR2738650B1; FR2738650A1; EP0850459A1

Abstract

An indicia device wherein the indiciae can be modified by handwriting and correspond to related symbolic or alphanumeric values. The indiciae are modified by entering simple bars in predetermined areas, and can be directly printed with their final values. The invention provides such a device, with a related method for analysing the image obtained by the optoelectronic scanning of the document bearing said indiciae. The method includes a system for analysing and detecting the sheet type, the areas with indiciae, the indiciae inside the areas, and any handwritten or printed changes to the base indiciae. It also includes a self-correcting technique for the optoelectronically scanned document images and a technique for learning the type and structure of the document contents. The method is useful for filling administrative documents and for repetitive checking applications, and for the fast and error-free computer acquisition thereof by optoelectronic scanning.

Description

SIGN DEVICE AND OPTICAL READING METHOD

DESCRIPTION

The invention relates to a device for alphanumeric and symbolic signs with manuscript or printed filling and to a method of recognition by optoelectronic reading of these signs or of any object contained in a document. Programs for the optical reading of documents containing handwritten alphanumeric characters are the subject of considerable research. To be effective, these programs require computer processing power, both in terms of hardware and software, which is very expensive, for reliability of the results which is still limited. Provided that the progress of these techniques is rapid? their application in the field of industrial and administrative information circulation is far from obvious.

Many technical documents with a handwritten alphanumeric filling must be re-entered by IT operators performing thankless tasks with equally imperfect reliability.

If the check boxes allow a certain reliability for the processing of binary information of the yes-no type by computer program, it should be noted that no simple, reliable handwritten system, using only a surface roughly equivalent to alphanumeric characters manuscripts are not available.

All the procedures used to date recognize the check box as an area in terms of white (empty) or gray (checked) value without identifying the empty or checked box, hence the difficulties of the associated programs. Apart from the delicate problem of setting the locations of the check boxes which must always be positioned in the same place in the mechanical and optoelectronic reading processes, this technique is very sensitive to the interference (or noise) of peripheral graffiti in the check boxes. Any document positioning incident results in serious data interpretation errors. Above all, these systems require 10 boxes per number, 26 per letter which makes filling them difficult, long to read and compromises their reliability.

This is why we are oriented towards a solution mixing the material design of the document, the sign and the computer program, by discarding the techniques described. Our invention allows complete reading reliability, by combining:

- a definition of sign ergonomically simple to fill in by hand,

- a reading system using an optoelectronic device (scanner, camera, LCD camera and any digital reading system), - an analysis process

- the type of leaves

- areas with these signs,

- signs inside these areas,

- handwritten or printed modifications to the basic signs. - a methodology:

- autocorrection of images of documents read optoelectronically,

- learning the type and structure of the document content. Our objective was to define:

- signs that can be modified by the agent responsible for manual entry; these modifications are particularly simple to carry out and come from almost universal cultural references,

- a completely original analysis system compared to conventional checkbox systems based on the data from the optoelectronic reading system.

- possibilities for self-checking and learning in the recognition and processing process.

The characteristics and advantages of the invention will be illustrated by the detailed description of at least one embodiment of ^the invention given as an indication and shown in Figures 1 to 24.

1 - The signs The structural and spatial composition of the sign:

Our signs are intended to be recognized as numerical value. They are made up on the principle of single or multiple zones which are intended to receive handwritten or printed bars. Our signs may include a. two, three or more zones (Zl, Z2...). These areas are intended to receive handwritten or printed indications in the form of bars. These zones are associated with a reference zone, or target zone (ZC) inseparable from the sign in its use by the recognition processing (FIG-1). For reasons of space saving and ergonomics, the target zone can be confused with another zone (FIG-2) but this is only a particular mode of use of our signs which present themselves thus.

Single-zone signs, known by the classic term of check boxes, are not original in themselves. The zone can take 2 values, 0 and 1 depending on whether it is empty or checked (FIG. 3). The target area is integrated into the box which performs 2 functions: handwritten information support and reference area (see below).

The 2-zone signs include (FIG. 4): - a zone representing the units (ZU) and intended to support from 0 to 4 bars, - another representing the value 5 (Z5) intended to support 0 or 1 single bar. The combination of these 2 zones allows any numbering between 0 and 9. The combination of this type of signs with 2 zones makes it possible to account for any value between 0 and infinity. The target zone (ZC) is merged with the value zone 5 which performs 2 functions. The three-zone signs (FIG. 5) are intended for the production of values between 0 and 49 as minimally as possible on the surface. The principle is the same for the unit area (ZU) and the value 5 (Z5), the additional area (ZD) is intended to support from 0 to 4 bars representing the tens. The target zone is also confused with the value zone 5. In this case the sign is a number between 0 and 49, it is isolated and cannot give rise to the sequential use of classical numbering.

Two complementary zones (FIG. 6) ZC 1 and ZC 2 can easily be added to these zones for additional meanings: negative sign, power ... the target zone always remaining merged with the value zone 5. The principle of the sign

The interest of this representation of digital data in this form, lies in the interpretation system by optical reading and that we have developed (see infra 2- The system of analysis of the device of signs). To be properly interpreted by our ^analysis system, and enable learning and self-control developed, signs must meet the following defined principles.

These signs must include an area containing the original point of ^the analysis of the sign. It is from this point that the analysis of the digital image of the sign is made. This reference point is the center of all research on the state of the pixels of its environment. The radial zones must allow the registration of bars perpendicular to the radius starting from the point of origin

The sign is therefore only the visual mnemonic aid to facilitate the registration of bars II is defined above all by its center and a double perimeter of analysis. The first perimeter defines the search start areas of the crossbars according to the search axes, the second the search end limit. This external perimeter is not equidistant from the center (FIG 7) If a research axis does not exist. the outline of these two perimeters coincide on this axis. In the forms of signs offered, the area of the point of origin is also an inscription area (value area 1 in the case of a check mark for the check boxes, 5 for the other signs), in order to make the sign more compact. and ^* in the ergonomics of use facihtei so it is a particular form of the general principle.

The proposed signs

The signs of our device can take different presentations based on the principles defined.

The signs of the check box type are presented in conventional form (FIG 8) and have only one simple zone. The numeric signs can be presented in two vertical or horizontal forms from a basic module (FIG 9), the position of the empty edge can be according to the direction, top or bottom, left or right The filling of the signs is done in an identical way whatever the orientation of the sign (FIG 10 and 1 1) The square box is worth 0 if it is empty, 5 if it has a bar, the unit area is in the axial extension of the box, a bar in the unit area is worth 1, two bars are worth 2, three bars 3, four bars 4

This simple system allows serial and sequential use, as illustrated by FIG. 4 A grouping of these signs can be used for filling in information whose values are limited to a maximum. 99. 999 (FIG 12) This system of handwritten filling signs has the advantage over handwritten numbers of being able to be incremented and allowing filling in several times For example for a control between 0 and 9, it suffices for the ^operator ensure that the number is less than or greater than 5, then in the latter case, adjust by incrementing the units. The use of erasable ink and eraser allows the marks to be corrected without prejudice to optoelectronic reading. The general system described above has variants of presentation and use. If numerical information is included, not between 0 and 9, but between 0 and 10, the same sign can be used (longer unit bar) with software control allowing the enumeration of an additional unit. In this case, the digit sign is isolated and cannot give rise to the sequential use of conventional numbering.

If numerical information is between 0 and 31 as for dates, times, alphabetical values, the sign can be completed by 2 small tens bars. These signs can be personalized by a letter or a logo characterizing the type of information to be worn, to help the operator in filling it out. This customization also allows a posteriori control and easy visual reading. Software control of the consistency of the information carried is ensured during computer processing. For this type of sign, the sign is also isolated, and cannot give rise to the sequential use of classic numbering. The signs representing dates can be put in special form (D / M / Y) as illustrated (FIG. 13).

The two short lines, here at the top, are intended to receive the handwritten bars of tens, 3 maximum for the 31 days of the month, 1 for the months, then then 2 standard numerical signs of our device for the years The signs representing hours can be put in the form h / m illustrated (FIG. 14).

The alphabetical signs can be produced from ^a conventional digital mapping modifying handwritten (or the printing directly) the sign of Figure 15. Figure 16 illustrates the codification of classical French alphabet.

The signs that can be defined on the basis of these principles are sufficiently simple and differentiated to allow rapid filling of administrative documents. These signs can be generated in the form of images, structures or computerized fonts in any application: word processor, spreadsheet. The signs are printed either in their blank forms, to be filled in by hand, or in filled printed forms (sheet code, constant references ...). In the latter case, the printed signs may include an indication of their value: 1, 2, 3, ..., A, B, C, D, ... indicated next to the sign (instead of α, h, m above).

2- The system for analyzing the device of signs

The sheets containing such signs are intended to be successively: - digitally read by optoelectronic reading system,

- an analysis of the digital image,

- recognition of the alphanumeric or symbolic value of these signs

- and a record of the information collected for computer processing. Computer processing of analysis and recognition, to be very fast, requires a prior definition of the sheets to be recognized, and learning the types of information found there.

These sheets and documents can include various zones: printed alphanumeric zones (FIG. 2θΦ), graphic zones characterized by any type of information treated as an image (FIG. 20®), checkbox zones and zones comprising our device of case signs (FIG. 20®). These areas can have mixed content. These areas will undergo appropriate treatment:

- character recognition for printed alphanumeric areas - image capture and processing for graphic areas

- specific treatment described below for the areas comprising our sign system, and in particular for the identification area of the sheet.

Each zone can be the subject of different types of treatment.

A- The sign device analysis process The type of device sign analysis is directly linked to the characteristics of our blank signs, and to their filling system. In the system ^of classical analysis of internal area in the check box (white pixels, black pixels), we substitute a spectral type analysis system from the center of the target reference box.

1 - The analysis of each sign begins from a predefined point whose coordinates are supposed to be inside the target box. Each sign is identified and positioned beforehand in the image (see below: Self-control and learning). The target box constitutes the starting point for the analysis of each sign.

2- The analysis of the sign continues with a check to ensure that the empty box is the box we are looking for and not a simply white area (which would imply a positioning error). If the box is empty (not crossed out), the number of white pixels found before the black pixels of its outline, in the vertical and horizontal directions, confirms the shape and the state (not crossed out) of the box.

If the box has internal black pixels, the state of the target reference box is recognized as crossed out. A global analysis is carried out to confirm its external outline (the box should not be largely covered by a black spot, a software alarm system allows this extreme case to be resolved).

In both cases, a calculation of the center of the box shape found is made, to refine the coordinates of the starting point of the following analysis process.

3- The starting point of the analysis being checked and recalculated, a search from the outline of the box is carried out (internal perimeter FIG. 7), pixel by pixel inside the area located between the 2 thin bars extending the target square (FIG. 22 ©). This analysis depends on the type of sign. Directions, search lengths of ^the analysis will depend on the sign chosen in the reference table for that location, when learning, and defined in the definition file of the sheet. The number of bars is recognized by interpretation of the succession of white and black pixels. The analysis stops at the end of each zone (the coordinates of which are contained in the sheet definition file).

A software likelihood check is performed by algorithm (example: no more than 3 tens bars for the sign relating to the days of the month). B- General learning specific to a sheet

This learning is stored by the program in a definition file specific to a type of sheet which will include all the information allowing the processing by optoelectronic reading and recognition of the information carried on this type of sheet

From an image of the blank model sheet (FIG 20) obtained by optoelectronic reading, we identify each general area to be recognized (Φ, ©. Φ @)

5 Each zone corresponds to a portion of the image of the sheet, defined by the coordinates of a virtual rectangle containing the zone These zones are indexed and their type of content - alphanumeric, printed, graphic, device of box signs or zones and check box, sheet identification code (FIG 20 ©), is associated with this index o The alphanumeric zones π and the graphic zones (D are treated according to the conventional methods of computer processing of such digital data

The reading program will produce a digital image of each sheet, this image will be immediately analyzed in conjunction with the definition file specific to the type of sheet. The specific processing of each zone will be launched sequentially to obtain a set of graphic information, alphanumeric which must be organized to be saved in a file

C- Learning an area containing our device

By selecting in the general image the zone comprising our device of 0 signs, this zone appears alone on the screen (FIG 21)

We define the analysis structure specific to each sign in the area We start from the first selected sign (it can be any sign in the area) by clicking with a pointing device (mouse, graphic table), in the closed box of the sign (or on a white point which is thus defined as the starting point of

25 the analysis of the sign-zone) (FIG. 22) The center of the closed box is automatically recalculated to produce the coordinates serving as origin for the analysis of its environment (FIG. 22 (D) according to the type of corresponding sign at this location We define the properties of the sign with reference to a table of signs (FIG 17 and

30 23)

- alphabetical, numeric, hour type,

- shape dimensions, bars of units, possible tens, on the left, on the right, - position of the starting point for the analysis of the sign, in relation to the virtual frame of the area). These properties are memorized by the program in the definition file. We then pass to the next sign in the desired order of positioning in the recording file of the interpreted data. The signs that have been trained are placed on a colored background (FIG. 24), in order to distinguish them from the signs that remain to be defined.

The recording of the data interpreted after digital reading for each sheet, includes all the interpreted information relating to the different types of zones. Each field of the record associates a type of zone and information contained in this zone (if these are differentiated). This is imperative for the zones comprising our device because the information relating to a sign can be found in the result file between 2 information relating to other zones. The sheets may have their own code (FIG. 20®) of the type of our device. It is the reading of this code that will trigger each time the process of accessing the sheet definition file (see below Self-checking of the sheet type). The training file or sheet definition file will therefore contain a set of data, ordered according to the user's needs, which will allow the program to produce a result file comprising a complex and heterogeneous series of data from: analysis of signs of our system, graphic images, and alphanumeric texts.

3 - The system of self-control in the process of reading, recognition, and processing

The analysis system makes it possible to recognize empty and full boxes and to ensure that empty or full boxes are the boxes sought. What all of these signs have in common is the target square (square, quadrilateral or closed shape). This box is the initial target. Each sign is identified and positioned by type of sheet in areas specific to our image system.

Self-checking the sheet type The indication of the type of sheet, and therefore of the sheet definition file, must be transmitted to the program, prior to any software analysis of the image obtained by optoelectronic reading. This indication can be transmitted to the program by the operator before each reading of one or more sheets of the same type. This method has the drawback of not allowing self-checking, because even in the case of a homogeneous composition in principle of a batch of sheets, certain sheets may belong to another type, or be poorly introduced during optoelectronic reading. It is much simpler to integrate in each sheet a code which is recognized immediately after scanning the image. This code with constant positioning on all the sheets read must therefore be integrated into the sheets. This code consists of a set of numerical signs (FIG. 20 ®).

The integration of this code allows a batch reading of sheets of heterogeneous types, each having a different sheet definition file, without any external intervention.

This code allows automatic self-checking of the sheet read, by associating its sheet definition file with it.

Self-checking of the sign device:

The problem of controlling the positioning of the starting point for the analysis of each sign is complex since a displacement of 2 or 3 tenths of a millimeter can generate general errors in reading the areas of our sign system. These errors can come from:

- the production of copies of sheets: printing, offset photocopies, biased, presenting zoom effects, - the optoelectronic reading system: imperfect introduction, poorly adjusted introducer, unequal image processing for mechanical or electronic reasons. A hardware device for locating and controlling the image is therefore necessary. A frame is drawn around the areas containing the signs to be analyzed (FIG. 22 Φ). These frames have on the model sheet (FIG. 20) a positioning, relative to each other and with each sign, perfectly defined. These boxes can include the zone or zones containing the signs to be analyzed, where not to include them, the relative location participating in the same logical procedure. Likewise, the location of signs can be identified from standardized header and footer frames for a number of documents.

By pointing during learning, on the angles of the frame, the coordinates of these angles are memorized relative to the edges of the image (FIG. 22 Φ). When reading the sheets reproduced from the model sheet, the image produced by the optoelectronic reading system returns the image of these frames. Their positions and coordinates in the image are searched and compared to those of the model sheet contained in the definition file. A vector transformation of these picture frames retranslates the entire positioning of the pixels of the image of the area to make them correspond to the reference positioning of the model sheet and correct the zoom and rotation effects. When using our self-checking process, the sheet definition file therefore contains the coordinates of the frame (s) in relation to the image (to speed up their search), and the coordinates of each point of origin of the analysis of our signs in relation to these frames.

An analysis of test sheets makes it possible to take into account the local deformation effects at the level of the optoelectronic reading system, and systematically corrects (if necessary) the preceding vector transformation. Filling the signs of such a document is as simple and fast as filling it with handwritten numbers and letters, its optoelectronic reading and its computer processing are almost instantaneous and error-free. These learning and self-checking methods can be used for reading documents comprising: sheet type code zones; various printed alphanumeric zones; graphic areas; checkbox zones or zones comprising the box sign or zone sign device; areas with any structured information to be analyzed from a specific point of origin.

Variants of our sign system and their analysis Our sign system is based on a closed target box, the location of which is carried out on the basis of a very precise learning system and self-checking process. With a high image resolution, it is possible to recognize a quantity of signs-equivalent-characters far greater than 2000 per A4 page with total reliability.

If the number of character-equivalent signs is less than this quantity, the constraints of our sign system and our learning, self-checking and analysis process, allowing total reading security, can be eased .

The closed target box, to allow self-control, can be transformed into a simple zone whose coordinates of the starting point of our spectral type analysis are defined (FIG. 18 / l (D, 12 Φ and FIG. 19 Φ ) The number of handwritten or printed bars contained in the zones assigned to the units Φ, and possibly to the zones of values 5 or tens Φ can be recognized according to our method If these zones are wide enough, our auto-correction system can s "prove useless because the risks of angular variations or by zoom effect leading to reading errors, decrease with the increase in the size of the signs and bars. Any system derived from our analysis process and our signs device , based on the analysis of the number of bars in predefined areas, must therefore be considered within the scope of our claims

The signs can be identified being located inside one or more frames (FIG. 23) or outside these frames; these frames may be non-existent (FIG. 17).

Example: the target cell or reference zone can consist of a cell in a table, other cells receiving the bars of tens or 5 units (FIG. 18). The target cell can be common to several zones corresponding to different signs. Of course, the invention is not limited to the embodiments described and shown for which other variants are possible in:

• the number of signs, boxes, zones

• the number of locating frames

• the number and type of objects analyzed and recognized on the document • the type of document, without departing from the scope of the invention.

Claims

1 - Set of signs which can be modified by hand and corresponding to alphanumeric values of a classic printing character as well as to any numerical value associated with symbols; characterized in that the modification of the signs is effected by the inscription of simple bars in defined zones corresponding to numerical values; and in that these signs can be used in the form of images, structures or typefaces for printing and be printed with their alphanumeric or symbolic values zero (without bar) or definitive (printed bars). 2 - set of signs, according to claim 1, characterized in that it comprises at least a first zone which can comprise from 0 to 4 bars each representing the number 1 and a second zone which can comprise a bar representing the number 5, the set of said two zones making it possible to write the digits from 0 to 9; these signs can be used sequentially to signify unlimited numerical values.

3- set of signs, according to claim 2, characterized in that it comprises complementary zones, associated with said two basic zones, also comprising bars, used in particular for the indication of tens, the sign of the number, the powers of the number. 4 - set of signs according to any one of claims 1 to 3, characterized in that the sign comprises a closed box, recognizable in its shape and dimensions, serving as an origin for the analysis of its environment, and in what this environment can be changed by adding handwritten or printed bars to signify an alphanumeric or symbolic value. 5 - set of signs, according to any one of claims 1 to 3. characterized in that the sign comprises an area identifiable by a point serving as an origin for the analysis of its environment, and in that this environment can be modified by adding handwritten or printed bars to signify an alphanumeric or symbolic value. 6 - Method for learning analysis of case signs, according to claim 4, characterized in that the coordinates of the center of the closed case are recalculated to serve as an origin for the analysis of its environment according to the type of sign corresponding to this location and in that the type of checkbox is defined during this training: definition of the coordinates of the checkbox relative to the image or frame of the area, definition of the dimensions of the target checkbox closed, of the type of sign compared to a table of reference of the signs.

7 - Method for learning the analysis of sign-zones, according to claim 5, characterized in that the zone is identified by a point serving as the origin of its environment, according to the type of sign corresponding to this location and in that the type of sign-zone is defined by learning: defining coordinates of the point serving as the origin to ^the analysis with respect to the picture or frame of the area, the sign of deviation from a reference table signs.

8 - Method for learning optoelectronic reading of documents of the kind comprising:

- sheet type code areas;

- various alphanumeric zones printed;

- graphic areas;

- checkbox zones or zones comprising the device of box signs or zone signs;

- areas containing any structured information to be analyzed from a precise point of origin; according to any one of claims 1 to 5, characterized in that it makes it possible to establish from the model document, the structure of the computer file resulting from the optoelectronic reading of the reproductions of the model document filled in by hand or in printed form, and in that it gives a possibility of recognition of the code of the type of the sheets read, allowing the reading and the automatic recognition in disorder of sheets of different types, by ensuring the possibility of logical control and a rerouting of the information read towards the results files intended to receive the information, with recording of the data interpreted after digital reading, for each sheet, comprising all the interpreted information relating to the different types of zones, each field of the recording being relative to a type of zone and to information contained in this zone. 9 - A method of analyzing sign-boxes according to claim 6 characterized in that the sign-box comprising a closed box, recognized in its shape, its dimensions and its state serving as an origin for the analysis of its environment according to the type of sign corresponding to this location, the center of the box closed to produce the coordinates used for the analysis, the type of box signs is defined during a preliminary training: definition of the coordinates of the box sign in relation to the image or frame of the zone, definition of the dimensions of the closed target box, of the type of sign in relation to a reference table of signs.

10 - A method of analysis of sign-zones according to claim 7 characterized in that the sign-zone being identified during learning by a point serving as an origin for the analysis of its environment, the analysis is conducted according to the type of sign corresponding to this location and determines the state of the surface close to the point of origin (presence or not of black pixels) then the state of the other areas of the sign, the type of sign-area being defined during '' preliminary learning: definition of the coordinates of the point used for the origin of the analysis in relation to the image or the frame of the area, of the type of sign in relation to a reference table of signs.

1 1 - Method for self-checking optoelectronic document reading comprising areas including the device of case signs according to claim 4, or any sign with precise positioning, characterized in that it involves the use of one or several printed frames relative to which the coordinates of the signs are referenced according to the methods of claims 9 and 10.