JP7700541B2 - Image processing device, image processing method, and program - Google Patents

Description

The present invention relates to an image processing device, an image processing method, and a program.

Conventionally, by restoring a document printed on a recording medium to the original electronic document data before printing, it has become possible to reuse the electronic document data, thereby improving convenience.
In view of this, a technology is known in which a printed document is scanned using a scanner, and after image area separation processing and the like, OCR (character recognition) is performed on the character image, and the document is converted into, for example, an Office Open XML Document format (Office conversion, Word conversion) that is adopted in Microsoft Office (registered trademark).

Patent Document 1 discloses a technique for detecting a circumscribing rectangle for each character image in image data, dividing the image data into a plurality of blocks, and setting them as provisional judgment regions, for the purpose of accurately detecting the font size of a character image from image data without providing a table for determining the font size for each character code. A frequency distribution is created for the height of the circumscribing rectangle of each character image, which is a component of each provisional judgment region, and a provisional judgment region having two or more valid circumscribing rectangle heights is regarded as a regular judgment region. A provisional judgment region that is not regarded as a regular judgment region is integrated with other adjacent judgment regions until it satisfies the conditions for a regular judgment region, and a font size is determined for each regular judgment region by using the maximum height of the circumscribing rectangle of the character images therein as a representative value.
In this way, in Patent Document 1, a circumscribing rectangle is obtained for each character, the font size is detected based on a predetermined criterion, and the font size is determined by using the maximum height of the circumscribing rectangle of the character image as a representative value.
For this reason, in Japanese Patent Laid-Open No. 2003-233699, there is a problem in that the font sizes of non-adjacent areas (cells) in a table printed on a recording medium using, for example, Excel or Word, cannot be unified.

In addition, in conventional character recognition processing, in font size calculation processing, the region (cell) for determining the font size from the image data of the document is the character or word in the cell adjacent to the cell of interest.
However, in documents printed with tabular cells in Excel (registered trademark) or Word (registered trademark), for example, there is a problem in that the font sizes of cells that are not adjacent to the focused cell, i.e., cells that are a certain distance away from the focused cell, cannot be unified.

One embodiment of the present invention has been made in consideration of the above, and its purpose is to unify the font sizes corresponding to character images in non-adjacent cells in a table printed on a recording medium.

In order to solve the above problem, the invention described in claim 1 is an image processing device that identifies a character code corresponding to a character image in each of a plurality of cells from image data related to a document containing character images in each of the plurality of cells, and determines a font size corresponding to the character code, and is characterized in that it comprises: a font size provisional determination unit that provisionally determines a font size of a focused cell based on character images in cells adjacent to the focused cell; a cell attribute detection unit that detects attributes that represent characteristics of the character image included in the focused cell; a font size group classification unit that classifies character images in all cells included in the image data based on attributes of the character image in the cell and the provisionally determined font size for the character image in the cell; and a font size final determination unit that determines a font size for each of the classification results based on a result of the classification by the font size group classification unit and the provisionally determined font size.

According to the present invention, it is possible to unify the font sizes corresponding to character images in non-adjacent cells in a table printed on a recording medium.

1 is a diagram illustrating an example of the configuration of a system including an image processing apparatus according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a hardware configuration of the image forming apparatus illustrated in FIG. 1 . FIG. 2 is a hardware configuration diagram of the PC shown in FIG. 1 . FIG. 4 is a diagram illustrating an example of a functional configuration of an image processing apparatus configured by the PC illustrated in FIG. 3 . 5 is a functional block diagram showing a detailed configuration of a font size determination unit 140 shown in FIG. 4. 6 is a flowchart showing an operation of determining a font size in the font size determination unit 140 shown in FIG. 5 . 11A to 11D are diagrams showing specific examples of methods for classifying cells based on cell attributes. FIG. 13 is a diagram illustrating an example of a processing result by a conventional image processing device in the form of a table. FIG. 11 is a table showing the recognition results obtained by the image processing device of the present invention.

Hereinafter, the present invention will be described in detail with reference to the embodiments shown in the drawings.
The present invention has the following configuration in order to unify the font sizes corresponding to character images in cells that are not adjacent to each other.
That is, the image processing device of the present invention is an image processing device that identifies a character code corresponding to a character image in each of a plurality of cells from image data related to a document containing character images in each of the plurality of cells, and determines a font size corresponding to the character code, and is characterized by comprising: a font size provisional determination unit that provisionally determines the font size of a focused cell based on character images in cells adjacent to the focused cell; a cell attribute detection unit that detects attributes that represent the characteristics of the character image contained in the focused cell; a font size group classification unit that classifies the character images in all cells included in the image data based on the attributes of the character images in the cell and the font size provisionally determined for the character images in the cell; and a font size determination unit that determines a font size for each classification result based on the results of the classification by the font size group classification unit and the provisionally determined font size.
With the above configuration, it is possible to unify the font sizes of character images in cells that are not adjacent to each other.
The features of the present invention described above will be described in detail with reference to the following drawings. However, unless otherwise specified, the components, types, combinations, shapes, relative arrangements, etc. described in the embodiments are merely illustrative examples and do not intend to limit the scope of the present invention.
The above-mentioned features of the present invention will be described in detail below with reference to the drawings.

<System Configuration>
FIG. 1 is a diagram illustrating an example of the configuration of a system including an image processing apparatus according to an embodiment of the present invention.
The system 1 includes an image forming apparatus 10 and two personal computers, a PC 80 and a PC 90, which are configured to transmit and receive data to and from each other via a communication network N.
The image forming apparatus 10 is an MFP (Multifunction Peripheral) equipped with a scanner unit 31 that reads an original sheet, a printer unit 32 that forms an image on the sheet, and a communication function. The scanner unit 31 reads an original sheet set in an ADF (Auto Document Feeder) if one is provided. The image forming apparatus 10 provides the user with each of the functions of copying, printing, scanning, and facsimile by using the scanner unit 31 and the printer unit 32. The image forming apparatus 10 also executes these functions by receiving instructions from a user via an operation panel 40 or from a PC 80 or PC 90 via a communication network N.

The image forming device 10 also has the function of extracting electronic data text from a scanned image read by the scanner unit 31 using optical character recognition (hereinafter simply referred to as "character recognition" or "OCR" as necessary) technology, and the function of creating an editable document (e.g. a Microsoft Word document) based on that text and the scanned image.
The PC 80 and the PC 90 are terminals (computers) used by general users who use the image forming apparatus 10 .
The image processing apparatus according to the present invention is composed of a PC 80 and a PC 90 shown in FIG.

<Hardware Configuration of Image Forming Apparatus>
FIG. 2 is a diagram showing an example of a hardware configuration of the image forming apparatus 10 shown in FIG.
As shown in FIG. 2, the image forming apparatus 10 includes a controller 20, a short-range communication circuit 60, an engine control unit 30, an operation panel 40, and a network I/F (Interface) 50.
The controller 20 includes a CPU 25a which is the main part of the computer, a system memory (MEM-P) 22, a north bridge (NB) 25b, a south bridge (SB) 24, an ASIC (Application Specific Integrated Circuit) 26, a local memory (MEM-C) 27 which is a storage unit, a HDD (Hard Disk Drive) controller 28, and a HD 29 which is also a storage unit. The NB 25b and the CPU 25a are integrated by a SoC (System on a Chip) 25.
The CPU 25a is a control unit that performs overall control of the image forming apparatus 10. The NB 25b is a bridge for connecting the CPU 25a with the MEM-P 22 and the SB 24, and includes a memory controller that controls reading and writing to the MEM-P 22, a PCI (Peripheral Component Interconnect) master, and an AGP target.

The MEM-P22 consists of a ROM22a, which is memory for storing programs and data that realize the various functions of the controller 20, and a RAM22b, which is used for expanding programs and data, and as a drawing memory during memory printing. The programs stored in the RAM22b may be provided by recording them in an installable or executable format on a computer-readable recording medium such as a CD-ROM, CD-R, or DVD.

SB24 is a bridge for connecting NB25b to PCI devices and peripheral devices. ASIC26 is an IC (Integrated Circuit) for image processing applications that has hardware elements for image processing, and plays the role of a bridge connecting PCI bus 23, HDD controller 28, and MEM-C27. This ASIC26 is composed of a PCI target and AGP master, an arbiter (ARB) that is the core of ASIC26, a memory controller that controls MEM-C27, multiple DMACs (Direct Memory Access Controllers) that rotate image data using hardware logic, and a PCI unit that transfers data between scanner unit 31 and printer unit 32 via PCI bus 23. Note that ASIC26 may be connected to a USB (Universal Serial Bus) interface or an IEEE1394 (Institute of Electrical and Electronics Engineers 1394) interface.

The MEM-C 27 is a local memory used as a copy image buffer and a code buffer. The HD 29 is a storage that stores programs for executing the operations described below and various control data in advance, and is also used to store image data, font data and forms used during printing. The HD 29 controls the reading and writing of data from and to the HD 29 under the control of the CPU 25a.
The short-distance communication circuit 60 also includes a communication circuit 60a for NFC, Bluetooth (registered trademark), or the like, and realizes data communication with an IC card held by the user. When the user holds the IC card close to the communication circuit 60a, the short-distance communication circuit 60 reads various data recorded on the IC card.

The engine control unit 30 controls the scanner unit 31 and the printer unit 32. The operation panel 40 includes a panel display unit 40a such as a touch panel that displays current settings and a selection screen and receives input from an operator, and an operation panel 40b including a numeric keypad that receives settings for image formation conditions such as density settings and a start key that receives a copy start command. The controller 20 controls the entire image forming apparatus 10, and controls, for example, drawing, communication, and input from the operation panel 40. The scanner unit 31 or the printer unit 32 includes an image processing unit such as error diffusion and gamma conversion.
The image forming apparatus 10 can successively switch between a document box function (scanner function), a copy function, a printer function, and a facsimile function by using an application switching key on the operation panel 40. The document box function (scanner function) also includes a function for extracting text from an image by OCR and converting it into a format that can be used by various office applications.
The network I/F 50 is an interface for performing data communication using the communication network N. The short-range communication circuit 60 and the network I/F 50 are electrically connected to the ASIC 26 via the PCI bus 23.

<PC hardware configuration>
Fig. 3 is a diagram showing the hardware configuration of the PC shown in Fig. 1. Here, the hardware configuration of the PC will be described.
As shown in FIG. 3, the PCs 80 and 90 are constructed by a computer, and as shown in FIG. 3, include a CPU 201, a ROM 202, a RAM 203, a HD 204, a HDD (Hard Disk Drive) controller 205, a display 206, an external device connection I/F (Interface) 208, a network I/F 209, a data bus 210, a keyboard 211, a pointing device 212, a DVD-RW (Digital Versatile Disk Rewritable) drive 214, and a media I/F 216.
Of these, the CPU 201 controls the overall operation of the PCs 80 and 90. The ROM 202 stores programs used to drive the CPU 201, such as IPL. The RAM 203 is used as a work area for the CPU 201. The HD 204 stores various data such as programs. The HDD controller 205 controls the reading and writing of various data from and to the HD 204 under the control of the CPU 201.

The display 206 displays various information such as a cursor, a menu, a window, characters, or an image. The external device connection I/F 208 is an interface for connecting various external devices. In this case, the external devices are, for example, a USB (Universal Serial Bus) memory or a printer. The network I/F 209 is an interface for data communication using the communication network N. The bus line 210 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 201 shown in FIG. 3.
The keyboard 211 is a type of input means having a plurality of keys for inputting characters, numbers, various instructions, etc. The pointing device 212 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The DVD-RW drive 214 controls reading and writing of various data from a DVD-RW 213, which is an example of a removable recording medium. Note that this is not limited to a DVD-RW, and may be a DVD-R or the like. The media I/F 216 controls reading and writing (storing) of data from a recording medium 215, such as a flash memory.

<Functional configuration of the image processing device>
FIG. 4 is a diagram showing an example of a functional configuration of an image processing apparatus constituted by the PC shown in FIG.
The image processing device 120 has the function of creating character codes from image data of a document and detecting font sizes, and is equipped with an image acquisition unit 125, an object recognition unit 130, a font size determination unit 140 and an electronic data generation unit 150. For example, the image processing device 120 is configured by installing a program for executing the processing described below on the HD 104 of a personal computer, and the CPU executing the program read from the HD 104.
In addition, the image processing device 120 may be configured by installing a program for executing the processing described below in the HD 29 of the image forming device 10 shown in Figure 2, and having the CPU 25a execute the program read from the HD 29.

The image acquisition unit 120 acquires image data of an original from an external terminal via a LAN or from a portable storage medium such as a USB memory. It may also be configured to acquire image data by reading an original including a document from the scanner unit 31 of the external image processing device 10 via a communication network N or by using an external camera.

The object recognition unit 130 expands the image data of the acquired document in a page memory and recognizes objects. Objects include characters, tables, pictures, figures, and the like. Known techniques are used as a means for recognizing objects in this way. For example, edge enhancement processing is performed on the image data, pixels are scanned vertically and/or horizontally to detect edges, and an area where the detection frequency is higher than a certain threshold can be determined as a character object. In addition, for table extraction, the table area can be extracted as a table object by the method described in Japanese Patent No. 5153857. Other areas can be determined as picture or figure objects.
A known character recognition process is performed on the image data of the character object to generate text data.
For example, for image data of a character object, density histograms in the horizontal and vertical directions are created, the image is cut out for each character, feature points (e.g., the number of closed loops or independent points and their positions) are extracted from each cut-out character image, and the character code is identified by matching with character patterns stored in a previously prepared table.

The object recognition unit 130 transmits the character code obtained by performing character recognition processing on the character object together with its position information as character code data to the electronic data generation unit 150. For table, picture, and graphic objects, the object recognition unit 130 transmits image data within the area and position information (memory address) to the electronic data generation unit 150. The object recognition unit 130 also transmits the character code data, image data of the character object, and image data of the table object to the font size determination unit 140.

The font size determination unit 140 determines the font size for each character based on the image data of the character object and the image data of the table object transmitted from the object recognition unit 130 .
The electronic data generation unit 150 associates the font size and character code of each character based on their position information so that the characters of the character code created by the object recognition unit 130 can be displayed in the font size determined by the font size determination unit 140, and integrates them with the table, picture, and graphic objects of the image data acquired by the image acquisition unit 120 to generate electronic data in a format that can be edited by a specified editing application.

Each function of the above-described embodiments can be realized by one or more processing circuits. In this specification, the term "processing circuit" includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, and devices such as an ASIC (Application Specific Integrated Circuit), DSP (digital signal processor), FPGA (field programmable gate array), and conventional circuit modules designed to execute each function described above.

<Configuration of Font Size Determination Unit>
FIG. 5 is a functional block diagram showing a detailed configuration of the font size determination unit 140 shown in FIG.
The font size determination unit 140 includes a tentative font size determination unit 141 , an in-table character determination unit 142 , a cell attribute detection unit 143 , a font size group classification unit 144 , and a final font size determination unit 145 .
The font size tentative determination unit 141 tentatively determines the font size for each character based on the image data of the character object received from the object recognition unit 130. For the means of tentatively determining the font size, for example, the technique described in Japanese Patent Laid-Open No. 2003-233996 is used.
In Patent Document 1, a circumscribing rectangle is obtained for each character, and the font size is detected based on a predetermined criterion. Furthermore, the font size is adjusted by taking into account information about characters (adjacent characters, words) surrounding the character of interest, thereby making it possible to unify the font size on a word-by-word or sentence-by-sentence basis.

The in-table character determination unit 142 determines whether a character is inside a table based on the position information of a character object and the position information of a table object.
The cell attribute detection unit 143 detects the attributes of the character image in each cell of the table object. The detected cell information includes at least the type of character system, such as English letters, numbers, and Japanese characters (kanji and kana), the type of character alignment, such as center alignment, left alignment, and right alignment, and the type of background color. The type of character system can be determined based on character code data. The cell attribute detection unit 143 can also determine the type of character alignment by creating a density histogram in the cell and determining the degree of bias. The cell attribute detection unit 143 calculates the average value of the RGB values of the part of the character image in the cell excluding the character part, and determines the color with the smallest Euclidean distance as the background color among multiple standard colors (10 colors) defined in Microsoft Office (registered trademark).

The font size group classification unit 144 classifies font sizes in the table object based on the attributes of the character images in each cell and the provisionally determined font size. The detailed classification method is shown in FIG.
The font size final determination unit 145 finally determines the font size based on the font size determined by the provisional font size determination unit, the in-table character determination result determined by the in-table character determination unit, and the font size group information classified by the font size group classification unit. If the character is not in a table, the provisionally determined font size is the final font size. If the character is determined to be in a table, the font size group information determines the most common font size among the provisionally determined font sizes of characters in the same classification. However, if there is a tie, the average size is used.

<Operation of Font Size Determination Unit>
FIG. 6 is a flowchart showing the operation of determining a font size in the font size determining unit 140 shown in FIG.
In step S10, the font size tentative determination unit 141 tentatively determines a font size based on the character object data and character code data received from the object recognition unit 130.
In step S20, the in-table character determination unit 142 determines whether or not the character object of interest is an in-table character, based on the character object data and the table object data received from the object recognition unit 130. If the determination result indicates that the character object is an in-table character, the process proceeds to S20, and if the character object is not an in-table character, the process proceeds to S30.

In step S30, after intra-table character determination unit 142 has completed determination of all character objects, cell attribute detection unit 143 detects cell attributes based on the intra-cell character objects and character code data of the character images in each cell.
The cell attribute detection unit 143 detects attributes that indicate characteristics of a character image included in a cell of interest. The cell attribute detection unit 143 uses any one of the type of character system, the type of background color, and the type of character justification as an attribute of the character image in the cell.
The cell attribute detection unit 143 assigns, as the type of background color, the color with the smallest Euclidean distance from among a number of standard colors defined in Microsoft Office (registered trademark) based on the color components of the character image in the cell of interest.

In step S40, the font size group classification unit 144 classifies each character object into which font size group it belongs, based on the provisionally determined font size and the cell attribute information.
The font size group classification unit 144 classifies all of the character images in the cells included in the image data based on the attributes of the character images in the cells and the font sizes provisionally determined for the character images in the cells.
The font size group classification unit 144 classifies the cells as the same if all cell attributes are present, the cells are adjacent, and the difference between the provisionally determined font sizes of the characters in the adjacent cells and the character of interest is less than a predetermined value.
The font size group classification unit 144 classifies the cells as the same if all cell attributes are present, the cells are adjacent, and the difference between the provisionally determined font sizes of the characters in the adjacent cells and the character of interest is less than a predetermined value.
The font size group classification unit 144 classifies characters that have not been classified in the classification operation as the same if the cell attributes of the character system type and justification type are the same, the cells are adjacent, and the difference between the provisionally determined font sizes of the characters in the adjacent cells and the character of interest is less than a predetermined value.
For characters that have not been classified in the classification operation, the font size group classification unit 144 determines the provisionally determined font size as it is as the font size of the target character.

In step S50, the font size final determination unit 145 unifies character objects classified into the same font size group into the most common font size in that font size group, and finally determines the font size.
The font size final determination unit 145 finally determines a font size for each classification result, based on the classification result by the font size group classification unit 144 and the provisionally determined font size.
The font size final determination unit 145 finally determines the most common font size among the font sizes for each result of classification by the font size group classification unit 144.
If the number of most common font sizes for each result of classification by the font size group classification section 144 is the same, the font size final determination section 145 finally determines the average font size.
In step S60, for character objects that are determined not to be within a table by the in-table character determination unit 142, the provisionally determined font size is determined as is as the final font size.

<Classification method for each cell>
7A to 7D are diagrams showing a specific example of a method for classifying character images in each cell based on the cell attributes.
FIG. 7A is a diagram showing an original in which image data is printed on a recording medium.
In Fig. 7(a), the names of each part of a typical table are arranged from left to right on the top row as follows: table temporal HST, table head HD1, table head HD3, and table head HD3. In the table shown in Fig. 7(a), the food name, protein, lipid, and carbohydrate are printed from left to right on the top row.
In addition, the names of the parts of a typical table are arranged from top to bottom in the leftmost column as follows: front head HST, front side HS1, front side HS2, front side HS3. In the table shown in Fig. 7(a), the food name, milk, thick milk, and low-fat milk are printed from top to bottom in the leftmost column.

FIG. 7B is a diagram showing the font of the document printed on the recording medium and the font size of the font.
Ideally, after the electronic data is generated, it will be in the state shown in Fig. 7(b). Note that in Fig. 7(b), the font size of the original font is noted in parentheses.

Fig. 7C is a diagram showing an example of a font size after it has been provisionally determined by the font size provisional determination unit 141. In Fig. 7C, the font size is also noted in parentheses. The underlined "_" in the parentheses shown in Fig. 7C indicates a portion where a difference appears with the font size of the original document.
In some parts (places marked with _), the font size of the original document cannot be accurately reproduced. When the font size tentatively determined by the font size tentative determination unit 141, the font size is determined taking into consideration only adjacent characters, so that the font sizes of character images in a cell can be unified.
However, since the font size of the character images in other cells is not taken into consideration, there are cases where the font size varies between cells. For example, as shown in Fig. 7(c), only the number "40" in the upper and lower columns is pt20, and the font size is different from the numbers above and below. In normal office documents, if the characters have the same character alignment type, the same character system type, and the same background color type, it is more natural and comfortable to the eye if the font size is the same.

<Classification rules>
Next, a description will be given of the rules for classifying font sizes into groups based on cell attributes and provisionally determined font sizes by the font size group classifying unit 144. The rules for classification are as follows.
The font size group classification unit 144 classifies character objects into font size groups according to the following stages [1] to [3].
[1] Hereinafter, character objects that satisfy the conditions are considered to be in the same font size group.
(a) The type of character system (English letters, numbers, Japanese characters), background color, and character justification are the same. (b) They are adjacent. (c) The difference in the provisionally determined font size between the character objects in the adjacent cells is less than a specified value.

[2] For character images in cells that were not classified in [1], character objects that satisfy the following conditions are grouped into the same font size group.
(D) The type of character system (English letters, numbers, Japanese) and the type of character justification are consistent. (E) They are adjacent. (F) The difference in the provisionally determined font size between the character objects in the adjacent cells is less than a specified value.

[3] Cells that are not classified under [1] and [2] are classified as other font sizes.
FIG. 7(d) is the result of classifying FIG. 7(c) according to the above classification rules.
The difference in font size between (c) of [1] and (f) of [2] is set to 2pt here. Groups I to IV are classified at the stage of [1].
Group V is classified at the stage [2], and group VI is classified at the stage [3]. For example, the character objects in group III in [1] have the same group, with the type of font system being numbers, the type of background color being light gray, and the type of justification being right justified. Comparing groups III and IV, the type of font system and the type of justification are the same, they are adjacent, and the difference in font size is less than 2pt, but because the types of background colors are different, groups III and IV are separate groups.

The font size final determination unit 145 unifies the size of each group to the most common font size. However, if the groups have the same number of sizes, the sizes of the groups are unified to the average size. For example, in the case of group I, the most common sizes are the same at 20, 16, and 18 pt, so the font size is corrected to 18 pt, which is the average size, and the font size is finally determined.

<Processing results using conventional technology>
FIG. 8 is a diagram showing, in the form of a table, an example of a processing result obtained by a conventional image processing device.
In the conventional image processing device, table J1 is generated as a processing result, and table J1 includes a frame J3 containing four cells arranged in the horizontal direction. Each of the four cells in frame J3 contains a character image with a different font size. In other words, according to the conventional processing result, character images with different font sizes are determined for each cell.
In conventional image processing devices, the area for determining the font size from the image data of a document is "adjacent" characters or words. Also, when the image data includes cells, the font size is determined on a cell-by-cell basis.

<Results of processing according to the present invention>
FIG. 9 is a diagram showing, in the form of a table, the recognition results obtained by the image processing apparatus of the present invention.
In the image processing apparatus of the present invention, table J5 is generated as a processing result, and table J5 includes a frame J7 containing four cells arranged in the horizontal direction. Each of the four cells in frame J7 contains character images with the same font size. In other words, according to the processing result of the present invention, character images with the same font size are determined for character images in cells of the same classification.

In the image processing apparatus of the present invention, character images in each cell in the same table are classified and the font size is determined for each classification, so that undesired discrepancies in font sizes within the same table can be eliminated.
For classification, any one of the following attributes of the character image in each cell is used: type of font, type of background color, or type of justification. Here, it is assumed that character images in cells with more similar attributes have the same font size.
The present invention has the following features when determining the font size of characters in a table format.
The font size of a character image in a single cell is provisionally determined, attributes of the character image in the single cell are detected, all cells are classified based on the attributes of the character images in the cells and the provisionally determined font size, and a font size for each classification is determined based on the classification and the font size of the single cell.
In short, the feature of this method is that the font size is determined by the font size of the character image in a single cell and the classification of the cell.
According to the present invention, each cell in a table is classified and the font size is determined for each classification, thereby eliminating undesired font size discrepancies within the same table. For classification, any one of the following attributes of each cell is used: type of character system, type of background color, and type of justification.

<Summary of the functions and effects of the embodiment>
<First aspect>
The image processing device 120 of this embodiment is an image processing device that identifies a character code corresponding to a character image in each of a plurality of cells from image data related to a document containing character images in each of the plurality of cells, and determines a font size corresponding to the character code, and is characterized by comprising: a font size provisional determination unit 141 that provisionally determines the font size of a focused cell based on character images in cells adjacent to the focused cell; a cell attribute detection unit 143 that detects attributes that represent the characteristics of the character image included in the focused cell; a font size group classification unit 144 that classifies the character images in all cells included in the image data based on the attributes of the character images in the cells and the provisionally determined font size for the character images in the cells; and a font size final determination unit 145 that determines a font size for each classification result based on the results of the classification by the font size group classification unit 144 and the provisionally determined font size.
According to this aspect, the font size of a focused cell is provisionally determined based on the character images in cells adjacent to the focused cell, attributes representative of the characteristics of the character images contained in the focused cell are detected, the character images in all cells contained in the image data are classified based on the attributes of the character images in the cells and the provisionally determined font size for the character images in the cells, and a font size for each classification result is determined based on the classification results and the provisionally determined font size.
This allows the font sizes of character images in non-adjacent cells to be unified.

<Second aspect>
The cell attribute detection unit 143 of this embodiment is characterized in that it uses any one of the type of character system, the type of background color, and the type of character justification as an attribute of the character image in the cell.
According to this embodiment, the cell attribute detection unit 143 can detect attributes that represent the characteristics of the character image contained in the cell of interest by using one of the type of font, the type of background color, and the type of character alignment as an attribute of the character image in the cell.

<Third aspect>
The cell attribute detection unit 143 of this embodiment is characterized in that it assigns the background color type to the color with the smallest Euclidean distance among multiple standard colors defined in Microsoft Office (registered trademark), based on the color components of the character image in the cell of interest.
According to this embodiment, the cell attribute detection unit 143 can detect attributes that represent the characteristics of the color components of the character image contained in the focused cell by assigning, as the type of background color, the color with the smallest Euclidean distance from among multiple standard colors defined in Microsoft Office (registered trademark) based on the color components of the character image in the focused cell.

<Fourth aspect>
The font size group classification unit 144 of this embodiment is characterized in that if all cell attributes are present, the cells are adjacent, and the difference between the provisionally determined font sizes of the characters in the adjacent cells and the character of interest is less than a predetermined value, the result is the same classification.
According to this embodiment, the font size group classification unit 144 can classify character images in cells by giving the same classification result if all cell attributes are present, the cells are adjacent, and the difference between the provisionally determined font sizes of the characters in the adjacent cells and the character of interest is less than a predetermined value.

<Fifth aspect>
The font size group classification unit 144 of this embodiment is characterized in that, for characters that were not classified in the classification operation, if the cell attributes of the character system type and justification type are consistent, the cells are adjacent, and the difference between the provisionally determined font sizes of the characters in the adjacent cells and the character of interest is less than a predetermined value, then the result is the same classification.
According to this embodiment, the font size group classification unit 144 can classify character images in cells by giving the same classification result to characters that were not classified in the classification operation if the cell attributes of the character system type and justification type are consistent, the cells are adjacent, and the difference between the provisionally determined font sizes of the characters in the adjacent cells and the character of interest is less than a predetermined value.

<Sixth aspect>
The font size group classification unit 144 of this embodiment is characterized in that, for characters that have not been classified in the classification operation, the provisionally determined font size is determined as the font size of the target character as is.
According to this embodiment, the font size group classification unit 144 can determine a font size for characters that were not classified in the classification operation by determining the provisionally determined font size as the font size of the target character as is, thereby determining a font size for characters that were not classified in the classification operation.

<Seventh aspect>
The main font size determination unit 145 of this embodiment is characterized in that it determines the most common font size among the font sizes for each classification result.
According to this aspect, the final font size determination unit 145 can determine the font size by determining the most common font size among the font sizes for each classification result.

<Eighth aspect>
The font size final determination unit 145 of this embodiment is characterized in that, when the most common font size among the font sizes for each classification result is the same, the font size final determination unit 145 determines the average font size.
According to this aspect, when the number of font sizes for each classification result that are the most common is the same, the font size final determination unit 145 can determine the font size by determining the average font size.

<Ninth aspect>
The image processing method of this embodiment is an image processing method for identifying character codes corresponding to character images in each of a plurality of cells from image data related to a document containing character images in each of a plurality of cells, and for determining a font size corresponding to the character code, and is characterized by comprising: a font size tentative determination step (S10) for tentatively determining a font size of a focused cell based on character images in cells adjacent to the focused cell; a cell attribute detection step (S30) for detecting attributes representative of characteristics of the character image included in the focused cell; a font size group classification step (S40) for classifying character images in all cells included in the image data based on the attributes of the character images in the cells and the font size provisionally determined for the character images in the cells; and a font size final determination step (S50) for determining a font size for each classification result based on the results of the classification by the font size group classification step (S40) and the provisionally determined font size.
According to this aspect, the font size of a focused cell is provisionally determined based on the character images in cells adjacent to the focused cell, attributes representative of the characteristics of the character images contained in the focused cell are detected, the character images in all cells contained in the image data are classified based on the attributes of the character images in the cells and the provisionally determined font size for the character images in the cells, and a font size for each classification result is determined based on the classification results and the provisionally determined font size.
This allows the font sizes of character images in non-adjacent cells to be unified.

<Tenth aspect>
The program of this aspect is characterized in that it causes a processor to execute each step of the image processing method according to the ninth aspect.
According to this aspect, each step can be executed by a processor.

1...system, 10...image processing device, 20...controller, 22a...ROM, 22b...RAM, 23...PCI bus, 25a...CPU, 26...ASIC, 28...HDD controller, 30...engine control unit, 31...scanner unit, 32...printer unit, 40...operation panel, 40a...panel display unit, 40b...operation panel, 50...network I/F, 60...short-range communication circuit, 60...short-range communication circuit, 60a...communication circuit, 80...PC, 90...PC, 101...CPU, 102...ROM, 103...RAM, 105...controller, 105...HDD controller , 106...display, 108...external device connection I/F, 109...network I/F, 120...image processing device, 125...image acquisition unit, 111...keyboard, 112...pointing device, 115...recording media, 116...media I/F, 120...image acquisition unit, 130...object recognition unit, 140...font size determination unit, 141...temporary font size determination unit, 142...table character determination unit, 143...cell attribute detection unit, 144...font size group classification unit, 145...formal font size determination unit, 150...electronic data generation unit, 208...controller

JP2016-062412A

Claims (10)

1. An image processing device that, from image data relating to a document including character images in each of a plurality of cells, identifies a character code corresponding to a character image in each of the plurality of cells, and determines a font size corresponding to the character code, comprising:
a font size provisional determination unit for provisionally determining a font size of a cell of interest based on character images in cells adjacent to the cell of interest;
a cell attribute detection unit that detects an attribute representing a feature of a character image included in the cell of interest;
a font size group classification unit that classifies character images in all cells included in the image data based on attributes of the character images in the cells and font sizes provisionally determined for the character images in the cells;
an image processing device comprising: a font size final determination unit that determines a font size for each result of the classification based on the result of the classification by the font size group classification unit and the provisionally determined font size. The image processing device according to claim 1, characterized in that the cell attribute detection unit uses one of the following attributes of the character image in the cell: type of font, type of background color, and type of character justification. The image processing device according to claim 2, characterized in that the cell attribute detection unit assigns, as the type of the background color, a color with the smallest Euclidean distance among multiple standard colors defined in Microsoft Office (registered trademark) based on the color components of the character image in the cell of interest. The image processing device according to claim 1, characterized in that the font size group classification unit classifies the characters into the same classification if all cell attributes are present, the cells are adjacent, and the difference between the provisionally determined font size of the character in the adjacent cell and the character of interest is less than a predetermined value. The image processing device according to claim 4, characterized in that the font size group classification unit classifies characters that have not been classified in the classification operation as the same classification result if the cell attributes of the character system type and justification type are the same, the cells are adjacent, and the difference between the provisionally determined font size of the characters in the adjacent cells and the character of interest is less than a predetermined value. The image processing device according to claim 4 or 5, characterized in that the font size group classification unit determines the provisionally determined font size as the font size of the target character for characters that are not classified in the classification operation. 2. The image processing apparatus according to claim 1, wherein the final font size determination unit determines the most common font size among the font sizes for each result of the classification. 2. The image processing apparatus according to claim 1, wherein, when the number of most common font sizes among the font sizes for each classification result is the same, the font size final determination unit determines an average font size. 1. An image processing method for identifying a character code corresponding to a character image in each of a plurality of cells from image data relating to a document including character images in the respective cells, and determining a font size corresponding to the character code, comprising:
a font size provisional determination step of provisionally determining a font size of the cell of interest based on character images in cells adjacent to the cell of interest;
a cell attribute detection step of detecting an attribute representing a feature of a character image included in the cell of interest;
a font size group classification step of classifying character images in all cells included in the image data based on attributes of the character images in the cells and font sizes provisionally determined for the character images in the cells;
a final font size determination step of determining a font size for each result of the classification based on the results of the classification in the font size group classification step and the provisionally determined font size. A program that causes a processor to execute each step of the image processing method described in claim 9.

Images