JP7767741B2 - Surface inspection device and program - Google Patents

Description

The present invention relates to a surface inspection device and program.

Today, various products use parts molded from synthetic resin (hereafter referred to as "molded products"). However, visually observable defects can appear on the surface of molded products. These defects include "sink marks," which are unintentionally formed depressions, and "welds," which form where molten resin meets. Even in the case of a textured finish, which intentionally creates unevenness on the surface, differences from the expected texture can appear. Texture changes depending on a combination of factors such as color, gloss, and unevenness.

JP 2009-264882 A

Some devices that inspect the surface quality of objects display the quality of the inspected part as a single number. Unlike sensory inspections, quantified numbers are highly objective. However, simply displaying a number does not reveal the main parts that contributed to the calculation of the number. As a result, workers cannot confirm whether the part they are focusing on is being used to calculate the number, or whether parts they did not expect are being used to calculate the number.

Unlike cases where the relationship between the numerical value representing surface quality and the parts that contributed to the calculation is unknown, the present invention aims to make it possible to confirm whether the parts that contributed to the calculation of the numerical value representing surface quality match the parts that the worker is focusing on.

The invention described in claim 1 is a surface inspection apparatus comprising an imaging device that images the surface of an object to be inspected, and a processor that processes the image captured by the imaging device and calculates a numerical value representing the quality of the surface corresponding to the entire inspection range of the captured image, wherein the processor displays the image, including an index that identifies the position of the part that contributed to the calculation of the numerical value, and the numerical value on a display device, and when there are multiple candidates for the numerical value, the processor displays the index that identifies the position of the part that corresponds to the largest numerical value.
A second aspect of the present invention is the surface inspection device according to the first aspect, wherein the index for specifying the position indicates a boundary between the portion and other portions.
A third aspect of the present invention is the surface inspection device according to the second aspect, wherein the index for specifying the position is a frame line that surrounds the periphery of the portion.
The invention described in claim 4 is the surface inspection device described in claim 1 , wherein, when there are multiple candidates for the numerical value, the processor displays the indicator that identifies the position of the part corresponding to each candidate.
The invention described in claim 5 is the surface inspection device described in claim 4 , wherein when a plurality of the indices are displayed, the processor associates the order of the magnitude of the corresponding numerical values with the display format of the indices.
The invention described in claim 6 is the surface inspection device described in claim 4 , wherein when a plurality of the indices are displayed, the processor switches the display of the numerical values and the indices in descending order of the magnitude of the corresponding numerical values.
A seventh aspect of the present invention is the surface inspection device according to any one of the first to sixth aspects, wherein the processor displays the numerical values alongside the corresponding indicators.
The invention described in claim 8 is a surface inspection device described in any one of claims 1 to 7 , wherein the processor notifies a user when the brightness difference in the image corresponding to the portion is greater than a predetermined threshold.
A ninth aspect of the present invention is a surface inspection device according to any one of the first to seventh aspects, wherein the processor displays the numerical value and the indicator on the image when inspection starts.
The invention described in claim 10 is a surface inspection device described in any one of claims 1 to 7 , wherein the processor displays the numerical value and the indicator on the image when it receives an operation to instruct the start of inspection.
An eleventh aspect of the present invention is the surface inspection device according to any one of the first to tenth aspects, characterized in that the device body is portable.
The invention described in claim 12 is a program for enabling a computer that processes an image of the surface of an object to be inspected captured by an imaging device to perform the following functions: calculate a numerical value representing the quality of the surface corresponding to the entire inspection range of the captured image ; display the image, including an index that identifies the position of the part that contributed to the calculation of the numerical value, and the numerical value on a display device; and, if there are multiple candidates for the numerical value, display the index that identifies the position of the part that corresponds to the largest numerical value .

According to the first aspect of the present invention, even when there are multiple candidates for the numerical value representing the surface quality, the position of the part corresponding to the numerical value displayed on the screen can be confirmed.
According to the second aspect of the present invention, it is possible to easily check the portion that contributed to the calculation of the numerical value.
According to the third aspect of the present invention, the content and position of the image of the portion that contributed to the calculation of the numerical value can be confirmed at the same time.
According to the fourth aspect of the present invention, when there are a plurality of candidates for the numerical value representing the surface quality, the position of the part related to the quality can be confirmed.
According to the fifth aspect of the present invention, it is possible to confirm the relationship between the magnitudes of the numerical values that represent the surface quality and the positions of the corresponding parts.
According to the sixth aspect of the present invention, the position of the portion corresponding to the numerical value representing the surface quality can be confirmed.
According to the seventh aspect of the present invention, the relationship between the numerical value representing the surface quality and the corresponding part can be easily confirmed.
According to the eighth aspect of the present invention, the user can be asked to confirm the calculated numerical value and the position of the part that contributed to the calculation.
According to the ninth aspect of the present invention, the operability during testing can be improved.
According to the tenth aspect of the present invention, the operability during testing can be improved.
According to the eleventh aspect of the present invention, even if the surface of the object to be inspected and the range imaged by the surface inspection device are indefinite, the accuracy of surface quality inspection can be improved.
According to the twelfth aspect of the present invention, even when there are multiple candidates for the numerical value representing the surface quality, it is possible to confirm the position of the part corresponding to the numerical value displayed on the screen .

1 is a diagram illustrating an example of use of a surface inspection device assumed in the first embodiment. FIG. 1A and 1B are diagrams illustrating examples of defects that appear on the surface of an inspection object, where (A) shows an example of a sink mark, and (B) shows an example of a weld. FIG. 2 is a diagram illustrating an example of a hardware configuration of a surface inspection device used in the first embodiment. 1A and 1B are diagrams illustrating an example of the structure of an optical system of a surface inspection device according to embodiment 1. FIG. 1A shows a schematic diagram of the internal structure of a housing of the surface inspection device, and FIG. 1B shows the structure of an opening that is pressed against the surface of an inspection target during inspection. 10 is a flowchart illustrating an example of an inspection operation by the surface inspection device used in the first embodiment. FIG. 2 is a diagram illustrating an example of an operation screen displayed on a display. 1A and 1B are diagrams illustrating the principle of score calculation, in which (A) shows an example image acquired as an object of inspection, (B) shows an example cross section in the x direction of a depression formed in the y-axis direction, and (C) shows the brightness profile of the image. FIG. 10 is a diagram illustrating an example of an operation screen on which the main contributing factors of the score are displayed. FIG. 10 is a diagram illustrating another example of the operation screen on which the main contributing factors of the score are displayed. 10A and 10B are diagrams illustrating an example of how a frame line is displayed when a plurality of stripe-like patterns are included in an image captured by a camera. 10A and 10B are diagrams illustrating another example of how a frame line is displayed when a plurality of stripe-like patterns are included in an image captured by a camera. 10A and 10B are diagrams illustrating another example of how a frame line is displayed when a plurality of stripe-like patterns are included in an image captured by a camera. 10A and 10B are diagrams illustrating other examples of border line display when multiple streak patterns are included, in which (A) is a screen displaying a border line indicating the main cause location of the maximum score, and (B) is a screen displaying a border line indicating the main cause location of the second largest score. 10A and 10B are diagrams illustrating examples of displaying scores that represent surface quality. 10A and 10B are diagrams illustrating other display examples of scores representing surface quality. 10A and 10B are diagrams illustrating an example of additionally displaying the score of a partial region designated by an operator, in which (A) is a diagram illustrating the designation of a partial region by an operator, and (B) shows an example of the display of an operation screen after the designation. 10A and 10B are diagrams illustrating other examples of indicators representing the main contributing areas used in calculating the score, in which (A) a symbol is displayed at a position that serves as a guide for the outer edge of the target partial area, and (B) an arrow represents the target partial area. 10A and 10B are diagrams illustrating other display examples of frame lines indicating partial regions that contributed to the calculation of the score. 11 is a flowchart illustrating an example of an inspection operation by the surface inspection device used in the second embodiment. FIG. 10 is a diagram illustrating an example of use of a surface inspection device assumed in a third embodiment.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
First Embodiment
<Example of using a surface inspection device>
FIG. 1 is a diagram for explaining an example of use of a surface inspection device 1 assumed in the first embodiment.
The imaging unit of the surface inspection device 1 used in the first embodiment is a so-called area camera, and the imaging range (hereinafter referred to as the "imaging range") is defined by a surface. Illumination (not shown) is configured to include a component that satisfies the specular reflection condition for the entire imaging range.

1, the imaging range includes the entire object to be inspected (hereinafter also referred to as "inspection target") 10. However, the imaging range may include only a portion of interest in the inspection target 10. In this embodiment, the inspection target 10 is assumed to be a molded product.
In the case of inspection using an area camera, the inspection is performed in a stationary state using the surface inspection device 1 and the inspection object 10. In other words, the inspection of the surface of the inspection object 10 is performed in a state where the surface inspection device 1 and the inspection object 10 do not move relative to each other.

1, the inspection object 10 is plate-shaped, but the shape of the inspection object 10 is arbitrary. For example, the inspection object 10 may have a shape with a curved surface, such as a sphere or cylinder, in addition to a polyhedron.
The actual inspection object 10 may have holes, notches, protrusions, steps, and the like.
The surface finish of the inspection object 10 may be untreated, mirror-finished, semi-mirror-finished, or textured.

The surface inspection device 1 inspects the surface of an inspection object 10 for defects and texture.
Defects include sink marks and welds. Sink marks are depressions on the surface that occur in thick parts or ribs, while welds are lines that occur where the leading edges of molten resin meet inside the mold. Defects also include scratches and dents caused by collisions.
Texture is a visual and tactile impression, and is influenced by the color, gloss, and roughness of an object's surface. Surface roughness includes lines that appear when cutting a mold. These types of lines are different from defects.

2A and 2B are diagrams illustrating examples of defects that appear on the surface of the inspection object 10. (A) shows an example of a sink mark, and (B) shows an example of a weld. In (A) and (B), the defect locations are indicated by dashed lines. There are four sink marks in (A) of FIG.
The surface inspection device 1 in this embodiment is not limited to inspection of defects and texture, but is also used to inspect surface stains.

The surface inspection device 1 generates an image that highlights defects on the surface of the inspection object 10, and also quantifies and outputs the results of texture evaluation.
The defects here are irregularities or streaks that appear on what should be a flat surface, such as sink marks or welds. The texture is evaluated using a numerical value (hereinafter also referred to as a "score"). The score is an example of a numerical value that represents the quality of the surface of the inspection object 10.

The score is calculated using, for example, multivariate analysis, which analyzes features that appear in the brightness distribution, such as a streak pattern that extends along the direction of the sink mark.
In addition, there is also a method using artificial intelligence to calculate the score. For example, by feeding an image captured by a camera into a learning model that uses deep machine learning to determine the relationship between the image of the defect and the score, a score for a partial area within the inspection range can be calculated.

1 is placed parallel to a plane defined by the X-axis and Y-axis, in which case the normal to the surface of inspection object 10 is parallel to the Z-axis.
On the other hand, the surface inspection device 1 is placed vertically above the inspection object 10. In other words, the optical axis of the optical system used by the surface inspection device 1 to image the inspection object 10 is set to be approximately parallel to the normal to the surface of the inspection object 10. Hereinafter, the conditions required for this optical axis will also be referred to as "imaging conditions."

At this time, the surface inspection device 1 is installed at a position that satisfies the imaging conditions. The surface inspection device 1 may be installed by being fixed to a specific member, or may be installed so as to be detachable from the specific member.
However, the surface inspection device 1 may be a portable device. If the surface inspection device 1 is portable, an operator can inspect any surface by holding the surface inspection device 1 in his/her hand, for example, and pointing the light receiving surface toward the inspection target 10.
In FIG. 1, the appearance of the surface inspection apparatus 1 is simplified and shown as a substantially rectangular parallelepiped for the purpose of explaining the positional relationship between the surface inspection apparatus 1 and the inspection object 10.

<Configuration of surface inspection device>
FIG. 3 is a diagram illustrating an example of the hardware configuration of the surface inspection device 1 used in the first embodiment.
3 includes a processor 101 that controls the operation of the entire apparatus, a ROM (Read Only Memory) 102 in which a BIOS (Basic Input Output System) and the like are stored, a RAM (Random Access Memory) 103 used as a work area for the processor 101, an auxiliary storage device 104 that stores programs and image data, a display 105 that displays an image of the surface of the inspection object 10 and information related to operations, an operation reception device 106 that receives operations from an operator, a camera 107 that images the surface of the inspection object 10, a light source 108 that illuminates the surface of the inspection object 10, and a communication IF (Interface) 109 used for communication with the outside. The processor 101 and each part are connected via a signal line 110 such as a bus.

The processor 101, the ROM 102, and the RAM 103 function as a so-called computer.
The processor 101 executes a program to realize various functions. For example, the processor 101 executes a program to calculate a score that evaluates the texture of the surface of the imaged inspection target 10, etc.
Image data obtained by capturing an image of the surface of the inspection object 10 is stored in the auxiliary storage device 104. The auxiliary storage device 104 may be, for example, a semiconductor memory or a hard disk drive. Firmware and application programs are also stored in the auxiliary storage device 104. Hereinafter, firmware and application programs will be collectively referred to as "programs."
In addition, the program that realizes the functions described in this embodiment and other embodiments described later can be provided by communication means, and can also be stored on a recording medium such as a CD-ROM and provided.

The display 105 is, for example, a liquid crystal display or an organic EL display, and displays an image of the entire inspection object 10 or a specific portion of the inspection object 10. The display 105 is also used to position the imaging range relative to the inspection object 10.
In the present embodiment, the display 105 is provided integrally with the device main body, but it may be an external device connected via the communication IF 109, or may be part of another device connected via the communication IF 109. For example, the display 105 may be the display of another computer connected via the communication IF 109.

The operation reception device 106 is composed of a touch sensor arranged on the display 105, physical switches, buttons, etc. arranged on the housing.
In the present embodiment, a power button and an image capture button are provided as examples of physical buttons. When the power button is operated, for example, the light source 108 is turned on and image capture by the camera 107 is started. When the image capture button is operated, a specific image that was being captured by the camera 107 at the time of operation is acquired as an image for inspection.

A device that integrates the display 105 and operation reception device 106 is called a touch panel. The touch panel is used to receive user operations using keys displayed in software (hereinafter also referred to as "soft keys").

In this embodiment, a color camera is used as the camera 107. The imaging element of the camera 107 is, for example, a CCD (=Charge Coupled Device) imaging sensor element or a CMOS (=Complementary Metal Oxide Semiconductor) imaging element.
Since a color camera is used as the camera 107, it is possible in principle to observe not only the brightness but also the color tone of the surface of the inspection object 10. The camera 107 is an example of an imaging device.

In this embodiment, a white light source is used as the light source 108. The white light source generates light in which light in the visible light range is evenly mixed.
In this embodiment, a parallel light source is used as the light source 108. A telecentric lens is arranged on the optical axis of the camera 107.

In this embodiment, the light source 108 is positioned at an angle such that the light component specularly reflected from the surface of the inspection object 10 is mainly incident on the camera 107 .
The communication IF 109 is configured with modules that comply with wired or wireless communication standards, such as an Ethernet (registered trademark) module, a USB (Universal Serial Bus), a wireless LAN, or the like.

<Structure of optical system>
4A and 4B are diagrams illustrating an example of the structure of the optical system of the surface inspection device 1 according to embodiment 1. FIG. 4A shows a schematic diagram of the internal structure of the housing 100 of the surface inspection device 1, and FIG. 4B shows the structure of the opening 111 that is pressed against the surface of the inspection target 10 during inspection.
The opening 111 has an opening 111A through which illumination light that illuminates the surface of the inspection object 10 and reflected light reflected on the surface of the inspection object 10 are input and output, and a flange 111B that surrounds the outer edge of the opening 111A.

4, the opening 111A and the flange 111B are both circular, but the opening 111A and the flange 111B may have other shapes, such as a rectangle.
The opening 111A and the flange 111B do not need to be similar in shape, and the opening 111A may be circular and the flange 111B may be rectangular.

The flange 111B is used to position the imaging direction of the surface inspection device 1 relative to the surface of the inspection object 10. In other words, the flange 111B is used to position the camera 107 and light source 108 relative to the surface being inspected. The flange 111B also serves to prevent or reduce the incidence of external or ambient light into the opening 111A.

The housing 100 shown in FIG. 4A has a structure in which two roughly cylindrical members are joined together, with a light source 108 attached to one member and a camera 107 and a processor 101 attached to the other member.
A display 105 and an operation reception device 106 are attached to the side of the housing 100 on the side where the camera 107 is attached.

The MTF (= Modulation Transfer Function) within the field of view of the camera 107 is generally uniform. As a result, there is little variation in contrast due to differences in position within the field of view, making it possible to faithfully capture images of the surface of the inspection object 10.

In the case of Figure 4(A), the normal to the surface of the flat plate-shaped inspection object 10 is indicated by NO. Also in Figure 4(A), the optical axis of the illumination light output from the light source 108 is indicated by L1, and the optical axis of the light specularly reflected on the surface of the inspection object 10 is indicated by L2. Here, the optical axis L2 coincides with the optical axis of the camera 107.

In reality, the surface of the inspection object 10 has irregularities due to its structure or design, curved surfaces, steps, joints, minute irregularities formed during the molding process, and the like.
Therefore, the normal N0 of the inspection target 10 may be the average value of the normals N0 of the region AR of interest within the inspection target 10 or the normal N0 of a specific position P of interest.

Furthermore, the normal line N0 of the inspection object 10 may be the normal line N0 of an average virtual surface or a representative portion of the inspection object 10.
4, the optical axis L1 of the illumination light output from the light source 108 and the optical axis L2 of the camera 107 are both attached at an angle θ with respect to the normal line N0. The angle θ is, for example, approximately 30° or approximately 45°.

<Inspection operation>
5 is a flowchart for explaining an example of an inspection operation by the surface inspection device 1 used in embodiment 1. The symbol S shown in the figure means a step.
The processing shown in FIG. 5 is realized through the execution of a program by the processor 101 (see FIG. 4).
In the surface inspection device 1 of this embodiment, the light source 108 (see FIG. 4) is turned on by operating the power button, and the camera 107 (see FIG. 4) starts capturing an image. The captured image is displayed on the display 105 (see FIG. 4).

Fig. 6 is a diagram illustrating an example of an operation screen 120 displayed on the display 105. The operation screen 120 shown in Fig. 6 includes a display field 121 for an image captured by the camera 107 (hereinafter referred to as a "captured image field"), a score field 122, a display field 123 for an image in which the features of a partial region that contributed to the calculation of the score are highlighted (hereinafter referred to as a "highlighted image field"), and a legend 124.
A distribution of brightness values, i.e., a grayscale image, is displayed in the captured image field 121. In the case of Fig. 6, a line 121A indicating the outer edge of the inspection range used in calculating the score is displayed.

6, the inspection range is the area surrounded by four lines 121A. A score representing the surface quality is calculated for the image within the inspection range.
A legend 124 is shown to the right of the captured image field 121. In the case of Fig. 6, the shading of the captured image field 121 corresponds to gradation values from "192" to "255".
In the case of the operation screen 120 shown in FIG. 6, since the score has not yet been calculated, the score field 122 is blank and no image is displayed in the highlighted image field 123 either.

Returning to the description of FIG.
In this embodiment, when an operator checking the image displayed on the display 105 operates the image capture button, the image to be used for evaluating the surface quality is determined.
Therefore, the processor 101, which has started the inspection operation by operating the power button, determines whether or not it has accepted the operation of the image capture button (step 1). Operating the operation button is an example of an operation to instruct the start of an inspection.
While a negative result is obtained in step 1, the processor 101 repeats the determination in step 1.

If a positive result is obtained in step 1, the processor 101 acquires an image to be used for the examination (step 2). Specifically, the image displayed on the display 105 at the time the image capture button was operated is acquired.
In the case of this embodiment, when the image capture button is operated, even if the camera 107 continues to capture images, updating of the images displayed in the captured image field 121 (see FIG. 6) is stopped.
Next, the processor 101 calculates a score using the brightness profile within the inspection range (step 3).

7A and 7B are diagrams illustrating the principle of score calculation, in which (A) shows an example image acquired as an object of inspection, (B) shows an example cross section in the x direction of a depression formed in the y-axis direction, and (C) shows the brightness profile S of the image.
The depression shown in Fig. 7B represents, for example, a sink mark. For convenience of explanation, Fig. 7B shows a cross-sectional shape of an isosceles triangle, but of course, this shape is only one example.

In this case, the luminance profile S shown in FIG. 7C is given as a change in the luminance value (hereinafter referred to as "representative luminance value") that represents each coordinate in the X-axis direction.
The representative luminance value here is given by the integral value of the luminance values of each pixel with the same x coordinate. A convex waveform of the luminance profile S indicates an area that is brighter than the surroundings, and a concave waveform indicates an area that is darker than the surroundings.

The score is calculated as the difference between the maximum and minimum values of the luminance profile S, for example.
The score depends on the width, height, depth, number, etc. of the irregularities formed on the surface. For example, even if the height of the protrusions or the depth of the recesses are the same, the score of a partial region in which a protrusion or recess with a greater width is formed will be higher.

Furthermore, even if the width of the convex or concave portion is the same, a partial region in which a higher convex portion or a deeper concave portion is formed will have a higher score. In this embodiment, a high score means poor quality.
In this embodiment, the partial region that contributes to the calculation of the score is defined as the region between the start point of the convex waveform and the end point of the concave waveform of the brightness profile S shown in FIG. 7C.

Returning to the description of FIG.
Once the score is calculated, the processor 101 extracts the partial region corresponding to the maximum score (step 4). The partial region here is the image portion within the inspection region that contributed to the calculation of the score. If multiple maximum scores are found, multiple partial regions are extracted. Basically, one partial region is extracted.
Next, the processor 101 displays a frame indicating the extracted partial region within the image (step 5).The processor 101 also generates an image in which the characteristics of the extracted partial region are emphasized (hereinafter referred to as an "enhanced image") and displays it separately (step 6).

In this embodiment, the processor 101 extracts a specific periodic component that appears in a specific direction from the extracted partial region, and generates an enhanced image by overlaying a feature image on the original image through inverse transformation of the extracted periodic component.
For example, two-dimensional DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), FFT (Fast Fourier Transform), etc. are used to extract the periodic components.
When converting the feature image back to the original image, the intensity component (i.e., brightness value) of each pixel is normalized by the maximum value to expand the range of gradation of the feature image. Also, by mapping color components to the intensity component of the feature image, it is possible to distinguish it from the original image, which is expressed in grayscale.

By displaying the highlighted image, it becomes possible to check the surface condition even when it is difficult to visually check minute structures in a grayscale image captured of the surface of the partial region for which the score was calculated.
In this embodiment, the generated highlighted image is displayed alongside the grayscale image captured by the camera 107 on the same operation screen.
In addition, the processor 101 displays the corresponding score on the operation screen 120 (see FIG. 6) and ends the process (step 7).

<Example of operation screen>
An example of a screen displayed when the surface inspection device 1 inspects the inspection object 10 will be described below.

<Screen example 1>
8 is a diagram illustrating an example of the operation screen 120 on which the main contributing factors of the score are displayed. In FIG. 8, the parts corresponding to those in FIG. 6 are denoted by the same reference numerals.
In the case of FIG. 8, a striped pattern of adjacent high and low brightness areas appears near the center of the captured image field 121.
The score field 122 on the operation screen 120 displays the numerical value "3.1" as the score calculated for this partial region.
Furthermore, a frame 125 indicating the area that contributed to the calculation of the score (hereinafter referred to as the "main contributing area") is displayed superimposed on the original image in the captured image field 121. In the case of Fig. 8, the frame 125 is a rough rectangle with rounded corners.

The display of this frame line 125 makes it possible to determine whether the partial area used to calculate the numerical value displayed in the score column 122 matches the partial area that the worker is focusing on.
The frame line 125 here is an example of an "indicator for identifying the position of the part that contributed to the calculation of the score."
A frame line 125 shown in FIG. 8 surrounds the periphery of the partial region that contributed to the calculation of the score, thereby providing a boundary between the partial region that contributed to the calculation of the score and other partial regions.

On the operation screen 120 shown in FIG. 8, a legend 126 regarding the frame line 125 is added below the captured image field 121.
8, an emphasized image of the area surrounded by a frame line 125 is displayed in emphasized image field 123 on operation screen 120. Therefore, even if it is difficult to confirm the surface condition of inspection object 10 from the image captured by camera 107 alone, it becomes possible to confirm the surface condition.

<Screen example 2>
9 is a diagram illustrating another example of the operation screen 120 on which the main contributing factors of the score are displayed. In FIG. 9, the same reference numerals are used to denote the parts corresponding to those in FIG.
In the case of FIG. 9, a frame 125 indicating the main factor location that contributed to the calculation of the score is displayed near the top of the captured image field 121.
The captured image field 121 shown in Fig. 9 includes a streak-like pattern in which high brightness and low brightness are adjacent to each other, but the brightness difference within the partial area surrounded by the frame line 125 is greater. For this reason, the frame line 125 is displayed near the top of the captured image field 121. The score is "5".

However, the portion surrounded by frame line 125 corresponds to the structural outer edge of inspection object 10 (see FIG. 1) and its background.
Therefore, the worker can realize that the score displayed on the operation screen is not the score for the part he or she is interested in. In this case, the worker can take a new image of the object of inspection, etc.

In addition, if a pattern of brightness differences exceeding a threshold is detected in the image, the processor 101 may output an error signal to notify that the captured image is not suitable for inspecting surface quality.
The processor 101 may also output an error signal when the calculated score exceeds a threshold value. For example, when the threshold value is "4" and a score of "5" is calculated, the processor 101 outputs an error signal.

If an error signal is output, the processor 101 may discard the image used to calculate the score and autonomously re-acquire another image. The processor 101 may also notify the operator that re-imaging is necessary. The notification may be made using a display or sound. In this case, the processor 101 may display the reason why re-imaging is considered necessary.

<Screen example 3>
10 is a diagram illustrating an example of how a frame line 125 is displayed when a plurality of stripe patterns are included in an image captured by the camera 107 (see FIG. 3). In FIG. 10, parts corresponding to those in FIG. 8 are denoted by the same reference numerals.
10, the image to be inspected includes two horizontal stripe patterns in the captured image field 121. In other words, there are two score candidates to be displayed in the score field 122 in FIG.

Even in this case, the processor 101 (see FIG. 3) displays the largest of the two scores in the score column 122. The processor 101 also displays a frame line 125 indicating the streak pattern that contributed to the calculation of the score displayed in the score column 122, superimposed on the image in the captured image column 121. Specifically, the processor 101 displays the frame line 125 surrounding the lower of the two streak patterns.
This display lets the worker know that the score in the score column 122 has been calculated from the pattern below.

<Screen example 4>
11 is a diagram illustrating another example of how the frame line 125 is displayed when a plurality of stripe patterns are included in an image captured by the camera 107 (see FIG. 3). In FIG. 11, the same reference numerals are used to denote parts corresponding to those in FIG. 8.
In the case of FIG. 11 as well, the captured image field 121 of the image to be inspected includes two horizontal stripe patterns.

Here, if the two scores calculated for the two partial regions corresponding to the two stripe patterns are the same value, two frame lines 125 are displayed in the captured image field 121 .
However, even when partial regions corresponding to scores where the difference between the maximum score and the second or subsequent score is smaller than the threshold value are displayed with a frame 125, the operation screen 120 shown in FIG. 11 appears.

Even if there are three or more scores where the difference between the maximum score and the second or subsequent scores is less than the threshold, the number of frame lines 125 displayed in the captured image field 121 may be limited to a predetermined number. For example, only two frame lines 125 indicating the partial area corresponding to the maximum score and the partial area corresponding to the second highest score may be displayed in the captured image field 121.

In the example of Fig. 11, the two frame lines 125 are displayed in the same format, so that an operator looking at the operation screen 120 shown in Fig. 11 can confirm that there are two partial areas with the same score or two partial areas with approximately the same score.
11, it is not possible to know whether the two scores corresponding to the two partial regions are the same or different values. Furthermore, if there is a difference in the scores, it is not possible to know which partial region corresponds to the score displayed in the score column 122.
In the case of FIG. 11 as well, the highlighted image field 123 displays the highlighted image of the partial region corresponding to the maximum score.

<Screen example 5>
Fig. 12 is a diagram illustrating another example of how the frame line 125 is displayed when a plurality of stripe patterns are included in an image captured by the camera 107 (see Fig. 4). In Fig. 12, parts corresponding to those in Fig. 11 are assigned the same reference numerals.
In the example screen shown in Fig. 12, differences in score size are expressed by different colors of frame lines 125. For example, frame line 125A surrounding the partial area corresponding to the maximum score is displayed in red, and frame line 125B surrounding the partial area corresponding to the second largest score is displayed in green. In the example of Fig. 12, differences in display color are expressed by different line types.

If it is possible to distinguish the difference in the magnitude of the scores, the brightness of frame line 125A and the brightness of frame line 125B may be changed, or only one of frame line 125A and frame line 125B may be displayed by flashing.
Also, the difference in the magnitude of the score may be expressed by the difference in the type of line, such as a double line or a dashed line. In other words, the difference in the magnitude of the score may be expressed by the difference in the display format.
12, an explanation of the frame 125B is added as a legend 126. This additional explanation allows the operator to know which of the partial areas indicated by the frame 125A and the partial area indicated by the frame 125B corresponds to the maximum score.

<Screen example 6>
13A and 13B are diagrams illustrating other examples of how the frame 125 is displayed when multiple streak patterns are included. (A) is a screen displaying the frame 125 indicating the main cause location of the maximum score, and (B) is a screen displaying the frame 125 indicating the main cause location of the second largest score.
In FIG. 13, parts corresponding to those in FIG. 11 are denoted by the same reference numerals.
The example screen shown in FIG. 13 is used when a plurality of scores greater than a predetermined threshold are found, or when a plurality of scores whose difference from the maximum score is less than the threshold are found.

In the case of FIG. 13, the position of the frame line 125 superimposed on the image captured by the camera 107 and the numerical value displayed in the score field 122 are switched sequentially.
For example, when the worker taps the captured image field 121, the screen of Figure 13 (A) switches to the screen of Figure 13 (B), and when the worker taps the captured image field 121 again, the screen of Figure 13 (B) switches to the screen of Figure 13 (A).

13A, the score field 122 of the operation screen 120 displays "3.1," and the legend 126 displays the phrase "Main contributing area for maximum score." Upon seeing this display, the worker knows that the score of the partial area corresponding to the frame line 125 displayed in the captured image field 121 is "3.1," and that this value is the maximum score within the inspection range.
13B, the score field 122 of the operation screen 120 displays "3," and the legend 126 displays the phrase "Main cause of the second highest score." Upon checking this display, the worker knows that the score of the partial area corresponding to the frame line 125 displayed in the captured image field 121 is "3," and that this value is the second highest score within the inspection range.

Note that the operation screen 120 shown in Figures 13(A) and 13(B) is designed to display two scores, but if there are three or more scores to display, the operation screen 120 will switch between three or more screens.
However, if the number of screens to be switched is greater than a predetermined threshold, a function can be provided that allows the user to return to the previous screen by performing a specific operation, thereby making the worker's confirmation work more efficient than when screens can only be switched sequentially.
Furthermore, the processor 101 may switch the screen every few seconds.

<Screen example 7>
14 is a diagram illustrating an example of displaying scores that represent surface quality, in which parts corresponding to those in FIG. 8 are denoted by the same reference numerals.
In the case of the operation screen 120 shown in FIG. 14, the score column 122 is not arranged below the highlighted image column 123 .
14, a score column 122A is displayed alongside the frame line 125. This display makes it easy for the operator to understand the relationship between the score and the corresponding partial area.
In the example of FIG. 14, the score column 122A is placed below the frame line 125, but it may be placed in another position, for example, on the right side, left side, or top side.

15 is a diagram illustrating another example of displaying scores that represent surface quality, in which parts corresponding to those in FIG. 14 are denoted by the same reference numerals.
In the case of Fig. 15, two frame lines 125 are displayed in the captured image field 121. In the case of Fig. 15, the display format of the two frame lines 125 is the same as in screen example 4 (see Fig. 11).
However, in the case of the operation screen 120 shown in Figure 15, the corresponding score column 122A is displayed alongside the frame line 125, so even if the display format of the two frame lines 125 is the same, it is possible to know the score calculated from the corresponding partial area.
In the case of FIG. 15, the score of the upper pattern is "3" and the score of the lower pattern is "3.1".

<Screen example 8>
16A and 16B are diagrams illustrating an example of additionally displaying the score of a partial area designated by an operator. (A) illustrates the designation of a partial area by an operator, and (B) shows an example of the display of the operation screen 120 after the designation. In Fig. 16, parts corresponding to those in Fig. 15 are assigned the same reference numerals.
16A assumes that the display 105 is a touch panel. The worker moves his/her fingertip 130 so as to surround a partial area in the display field 121 for which the worker wants to check the score, thereby indicating to the processor 101 the partial area for which the worker wants to calculate the score.

The legend 126 in Fig. 16(B) indicates that the frame line 125C is the location designated by the operator. In Fig. 16(B), the frame line 125C is displayed in a color different from that of the frame line 125 displayed by the processor 101. In Figs. 16(A) and 16(B), the difference in color is expressed by the difference in line type.
In the case of Fig. 16(B), the score of the part specified by the worker is "3." This function allows the worker to freely check the score of any part that concerns them.
The point where the score is to be calculated can be specified not only by specifying the outer edge of the corresponding partial area but also by tapping on the point of interest. In this case, the processor 101 may determine the partial area including the tapped point and calculate the score.

<Screen example 9>
17A and 17B are diagrams illustrating other examples of indicators representing the main contributing areas used in calculating the score. (A) shows a case where a symbol is displayed at a position that serves as a guide for the outer edge of the target partial area, and (B) shows a case where the target partial area is represented by an arrow. In Fig. 17, the same symbols are used to denote parts corresponding to those in Fig. 6.
17A, triangular symbols 125D are displayed at the four corners of the main cause areas used in calculating the score. This display also allows the operator to know the partial areas used in calculating the score. Note that the shape of the symbols 125D is an example, and other shapes such as a circle or a star may also be used.

On the other hand, on the operation screen 120 shown in FIG. 17(B), the approximate position of the main factor part used in calculating the score is indicated by an arrow 125E.
The display using the arrow 125E does not reveal the extent of the main cause location. However, if the outer edge of the structure is shown, as in the case of screen example 2 (see FIG. 9), it becomes clear that the partial area used to calculate the score is different from the area the worker is focusing on.

<Screen example 10>
18 is a diagram showing another example of displaying the frame 125 indicating the partial region that contributed to the calculation of the score. In Fig. 18, the same reference numerals are used to denote the parts corresponding to those in Fig. 8.
In the case of the operation screen 120 shown in FIG. 18, a sink mark having a curved shape is displayed in the image field 121 captured by the camera 107 (see FIG. 4).
While the frame line 125 in the other screen examples is roughly rectangular, the frame line 125 shown in Fig. 18 is curved. That is, the shape of the frame line 125 is not limited to a roughly rectangular shape, but can take various shapes depending on the shape of the defect.

<Second Embodiment>
In this embodiment, a surface inspection device 1 (see FIG. 1) that does not require operation of an image capture button when calculating a score will be described.
The external configuration of the surface inspection device 1 in this embodiment is the same as that of the surface inspection device 1 described in the first embodiment.

Fig. 19 is a flowchart for explaining an example of an inspection operation by the surface inspection device 1 used in embodiment 2. In Fig. 19, parts corresponding to those in Fig. 5 are assigned the same reference numerals.
In the case of Figure 19, the processor 101 (see Figure 4) turns on the light source 108 (see Figure 4) when the power button is operated, starts capturing images with the camera 107 (see Figure 4), and simultaneously performs score calculations, etc.

For this purpose, the processor 101 acquires an image being captured by the camera 107 (step 11), and calculates a score using the brightness profile within the inspection range (step 3).
Thereafter, the processor 101 extracts the partial area corresponding to the maximum score (step 4), and performs the following operations (steps 5 to 7): displaying a frame line 125 (see Figure 8) indicating the extracted partial area, displaying an image of the corresponding partial area, displaying the corresponding score, etc.

After step 7 is completed, processor 101 returns to step 11 and repeats the series of processes. By repeating this series of processes, the score displayed on operation screen 120 (see FIG. 8 ) continues to be updated in accordance with changes in the image captured by camera 107.
That is, when the position at which the camera 107 captures an image changes, the positions of the score and the frame line 125 also change.

<Third Embodiment>
Fig. 20 is a diagram for explaining an example of use of the surface inspection apparatus 1A assumed in the embodiment 3. In Fig. 20, parts corresponding to those in Fig. 1 are assigned the same reference numerals.
The surface inspection apparatus 1A used in this embodiment uses a so-called line camera as the imaging unit, and therefore the imaging range is linear.

In the case of this embodiment, during inspection, the inspection object 10 is moved in the direction of the arrow while being placed on the uniaxial stage 20. By moving the uniaxial stage 20 in one direction, the entire inspection object 10 is imaged.
Note that, except that a line camera is used as the camera 107 (see FIG. 4), the positional relationship between the camera 107 (see FIG. 4) and the light source 108 (see FIG. 4) and the like are the same as those in the first embodiment.

<Other embodiments>
(1) Although the embodiments of the present invention have been described above, the technical scope of the present invention is not limited to the scope of the above-described embodiments. It is clear from the claims that various modifications and improvements to the above-described embodiments are also included in the technical scope of the present invention.

(2) In the above-described embodiment, a color camera was used as the camera 107 (see Figure 4), but a monochrome camera may also be used. Furthermore, only the green (G) component of the color camera may be used to inspect the surface of the inspection object 10 (see Figure 1).

(3) In the above embodiment, a white light source is used as the light source 108 (see FIG. 4), but the color of the illumination light may be any color.
Furthermore, the illumination light is not limited to visible light, but may be infrared light, ultraviolet light, or the like.

(4) In the above embodiment, the surface inspection device 1 (see FIG. 1) uses one light source 108 (see FIG. 4). However, the surface of the inspection object 10 may be illuminated using multiple light sources.
For example, two light sources may be used. In this case, one light source may be arranged at an angle where specularly reflected light components are mainly incident on camera 107 (see FIG. 4 ), and the other light source may be arranged at an angle where diffusely reflected light components are mainly incident on camera 107. In this case, the two light sources may be arranged on either side of the optical axis of camera 107, or may be arranged side by side on one side of the optical axis of camera 107.

(5) In the above-described embodiment, a parallel light source was used as the light source 108 (see Figure 4), but a point light source or surface light source, which is a non-parallel light source, may also be used. Furthermore, a non-telecentric lens may be used on the optical axis of the camera 107 (see Figure 4). When a telecentric lens or parallel light is not used, the device can be made smaller and less expensive than the surface inspection device 1 (see Figure 1) described in the embodiment.

(6) In the above-described embodiment, the processor 101 (see Figure 3) of the surface inspection device 1 (see Figure 1) that captures the image of the inspection object 10 (see Figure 1) realizes the function of calculating the score and displaying a frame 125 (see Figure 8) indicating the main partial area that contributed to the score calculation on the operation screen 120 (see Figure 8). However, equivalent functions may also be realized by a processor of an external computer or server that acquires image data from the surface inspection device 1.

(7) The processor in each of the above-described embodiments refers to a processor in a broad sense, and includes general-purpose processors (e.g., CPUs, etc.) as well as dedicated processors (e.g., GPUs (Graphical Processing Units), ASICs (Application Specific Integrated Circuits), FPGAs (Field Programmable Gate Arrays), programmable logic devices, etc.).
Furthermore, the operations of the processors in each of the above-described embodiments may be performed by a single processor alone, or may be performed by multiple processors located in physically separate locations in cooperation with each other. Furthermore, the order in which the operations of the processors are performed is not limited to the order described in each of the above-described embodiments, and may be changed individually.

1, 1A...Surface inspection device, 10...Inspection object, 20...One-axis stage, 101...Processor, 102...ROM, 103...RAM, 104...Auxiliary storage device, 105...Display, 106...Operation reception device, 107...Camera, 108...Light source, 109...Signal line, 111...Opening, 111A...Opening, 111B...Flange, 120...Operation screen, 122, 122A...Score column, 125, 125A, 125B, 125C...Border line, 125D...Symbol, 125E...Arrow

Claims (12)

an imaging device for imaging the surface of an object to be inspected;
a processor that processes the image captured by the imaging device and calculates a numerical value representing the quality of the surface corresponding to the entire inspection range of the captured image ;
the processor displays the image including an indicator specifying the position of the portion that contributed to the calculation of the numerical value and the numerical value on a display device;
When there are multiple candidates for the numerical value, the processor displays the indicator that identifies the position of the portion corresponding to the largest numerical value.
Surface inspection equipment. the indicator for identifying the position indicates a boundary between the portion and other portions;
The surface inspection device according to claim 1 . The indicator for identifying the position is a frame line surrounding the periphery of the portion.
The surface inspection device according to claim 2 . When there are a plurality of candidates for the numerical value, the processor displays the indicator that identifies the position of the portion corresponding to each candidate.
The surface inspection device according to claim 1 . When a plurality of the indices are displayed, the processor associates the order of magnitude of the corresponding numerical values with the display format of the indices.
The surface inspection device according to claim 4 . When the processor displays a plurality of the indicators, the processor switches the display of the numerical values and the indicators in descending order of magnitude of the corresponding numerical values.
The surface inspection device according to claim 4 . The processor displays the numerical value alongside the corresponding indicator.
The surface inspection device according to any one of claims 1 to 6 . the processor notifies a user when a luminance difference in the image corresponding to the portion is greater than a predetermined threshold.
The surface inspection device according to any one of claims 1 to 7 . the processor displays the numerical value and the indicator on the image when the examination begins.
The surface inspection device according to any one of claims 1 to 7 . When the processor receives an operation to instruct the start of the examination, the processor displays the numerical value and the indicator on the image.
The surface inspection device according to any one of claims 1 to 7 . The surface inspection device according to any one of claims 1 to 10 , characterized in that the device itself is portable. A computer processes an image of the surface of an object to be inspected taken by an imaging device.
a function of calculating a numerical value representing the quality of the surface corresponding to the entire inspection range of the captured image ;
a function of displaying the image including an index specifying the position of the portion that contributed to the calculation of the numerical value and the numerical value on a display device;
a function of displaying the indicator that identifies the position of the portion corresponding to the largest numerical value when there are multiple candidates for the numerical value;
A program to achieve this.