JP7703773B2 - Three-dimensional measurement calculation device, three-dimensional measurement program, recording medium, three-dimensional measurement device, and three-dimensional measurement calculation method - Google Patents

Description

This invention relates to a technology for measuring the three-dimensional shape of an object to be measured based on a pattern image obtained by capturing an image of a predetermined pattern of light irradiated onto the object to be measured.

Patent Documents 1 and 2 describe a three-dimensional measurement technique that uses a so-called phase shift method to measure the three-dimensional shape of an object to be measured. In this three-dimensional measurement technique, the three-dimensional shape of the object to be measured is measured based on an image obtained by using a camera to capture a predetermined pattern of light irradiated onto the object to be measured from a projector.

Patent Document 2 also proposes a technology for measuring three-dimensional shapes by suppressing the effects of secondary reflections that occur when a tall object is present among the objects constituting the measurement target. In other words, if secondary reflections occur in which light emitted from a projector and reflected off the side of a tall object is further reflected by another object, it becomes difficult to accurately measure the three-dimensional shape. Therefore, in Patent Document 2, the range of light emitted from the projector is limited so that the light from the projector does not enter a tall object (causing object) that causes secondary reflections.

JP 2012-112952 A JP 2016-130663 A

However, in order to limit the range of light emitted from the projector as in Patent Document 2, it is necessary to control hardware such as the projector equipped in the three-dimensional measurement device, which can make the control complex. In addition to the above-mentioned secondary reflection, other light irradiation problems that affect the measurement of three-dimensional shapes include so-called occlusion. This occlusion is a phenomenon in which a shadow is created when the light from the projector does not reach the object being measured due to being blocked by an object that constitutes the measurement target. Patent Document 2 cannot deal with this type of occlusion.

On the other hand, if it is possible to know the poorly irradiated areas on the measurement object where poor light irradiation occurs, the poorly irradiated areas can be referenced in the calculations that calculate the three-dimensional shape based on the pattern image acquired by the three-dimensional measuring device. As a result, it is possible to execute the calculations that calculate the three-dimensional shape while suppressing the effects of poor light irradiation.

This invention has been made in consideration of the above-mentioned problems, and aims to make it possible to obtain poorly irradiated areas where poor light irradiation occurs when measuring the three-dimensional shape of a measurement object based on a pattern image obtained by capturing an image of a predetermined pattern of light irradiated onto the measurement object.

The three-dimensional measurement calculation device of the present invention is a three-dimensional measurement calculation device that performs calculations to measure the three-dimensional shape of the measurement object based on a pattern image acquired by a three-dimensional measurement device that has an object support section that supports a measurement object having a substrate and a component mounted on the substrate, a pattern irradiation section that irradiates a predetermined pattern of light onto the measurement object supported on the object support section, and an imaging section that acquires a two-dimensional pattern image by imaging the light irradiated onto the measurement object from the pattern irradiation section.The three-dimensional measurement calculation device is equipped with a reference model acquisition section that acquires a reference model that indicates a component mounting range on the substrate where components are mounted and the outer shapes of the components to be mounted in the component mounting range, and an area calculation section that calculates at least one of the poorly irradiated areas among a shadow area where a shadow of the component is cast on the substrate due to the light irradiated from the pattern irradiation section being blocked by the component and a secondary reflection area where light irradiated from the pattern irradiation section and reflected by the component is incident on the substrate, based on irradiation direction information that indicates the direction in which light is irradiated onto the component by the three-dimensional measurement device and the reference model.

The three-dimensional measurement program of the present invention causes a computer to function as the above-mentioned three-dimensional measurement calculation device.

The recording medium of the present invention records the above-mentioned three-dimensional measurement program in a manner that allows it to be read by a computer.

The three-dimensional measuring device according to the present invention comprises an object support section that supports a measurement object having a board and components mounted on the board, a pattern irradiation section that irradiates a predetermined pattern of light onto the measurement object supported by the object support section, an imaging section that obtains a two-dimensional pattern image by imaging the light irradiated onto the measurement object from the pattern irradiation section, and the above-mentioned three-dimensional measuring calculation device that performs calculations to measure the three-dimensional shape of the measurement object based on the pattern image.

The three-dimensional measurement calculation method according to the present invention is a three-dimensional measurement calculation method that performs calculations to measure the three-dimensional shape of the measurement object based on a pattern image acquired by a three-dimensional measurement device that includes an object support section that supports a measurement object having a substrate and a component mounted on the substrate, a pattern irradiation section that irradiates a predetermined pattern of light onto the measurement object supported on the object support section, and an imaging section that acquires a two-dimensional pattern image by imaging the light irradiated onto the measurement object from the pattern irradiation section, and includes a step of acquiring a reference model that indicates a component mounting range on the substrate where the components are mounted and the outline of the components to be mounted in the component mounting range, and a step of calculating at least one of the poorly irradiated areas among a shadow area where a shadow of the component is cast on the substrate due to the light irradiated from the pattern irradiation section being blocked by the component and a secondary reflection area where light irradiated from the pattern irradiation section and reflected by the component is incident on the substrate, based on irradiation direction information that indicates the direction in which light is irradiated onto the component by the three-dimensional measurement device and the reference model.

In the present invention (three-dimensional measurement calculation device, three-dimensional measurement program, recording medium, three-dimensional measurement device, and three-dimensional measurement calculation method) configured in this manner, a reference model is acquired that indicates the component mounting area on the board where components are mounted and the outline of the components to be mounted in the component mounting area. Then, poorly irradiated areas are calculated based on the reference model and irradiation direction information that indicates the direction in which light is irradiated onto the components by the three-dimensional measurement device. In this way, it is possible to acquire poorly irradiated areas where poor light irradiation occurs when measuring the three-dimensional shape of the measurement object based on a pattern image acquired by capturing an image of a predetermined pattern of light irradiated onto the measurement object.

Here, the poorly irradiated area is at least one of a shadow area where light irradiated from the imaging unit to the component is blocked by the component, causing a shadow of the component on the board, and a secondary reflection area where light irradiated from the imaging unit and reflected by the component enters the board.

The three-dimensional measurement calculation device may be configured so that the reference model acquisition unit creates a reference model from at least one of the following data: inspection data indicating a standard for inspecting the suitability of the positional relationship between the component mounted in the component mounting area and the component mounting area, and CAD (Computer-Aided Design) data indicating the configuration of the board. In this configuration, a reference model is created using existing data such as the inspection data or CAD data of the board, and the poorly irradiated area can be calculated based on this reference model. Note that the information indicated by the inspection data is not limited to the above standards, and may also include standards for inspecting the suitability of the component itself and standards for inspecting the suitability of the bonding between the component and the board.

The three-dimensional measurement arithmetic device may also be configured so that the area calculation unit calculates the poorly irradiated area based on the results of recognizing the position of the substrate carried into the three-dimensional measurement device and supported by the object support unit. In this configuration, the poorly irradiated area can be accurately calculated according to the actual position of the substrate carried into the three-dimensional measurement device.

The three-dimensional measurement calculation device may be configured so that the position of the substrate supported by the object support unit is recognized based on the results of imaging the fiducial mark attached to the substrate by the imaging unit. In this configuration, the poorly irradiated area can be accurately calculated according to the position of the substrate carried into the three-dimensional measurement device.

The three-dimensional measurement calculation device may also be configured so that the area calculation unit calculates the poorly irradiated area based on the results of detecting the positions of the components mounted in the component mounting area. In this configuration, the poorly irradiated area can be accurately calculated according to the actual positions of the components mounted in the component mounting area.

The three-dimensional measurement calculation device may be configured so that the position of the component mounted in the component mounting area is detected based on the result of imaging the component. In this configuration, the poorly irradiated area can be accurately calculated according to the actual position of the component mounted in the component mounting area.

The three-dimensional measurement arithmetic device may be configured such that the pattern irradiation unit has a plurality of projectors that irradiate the component with light from different directions, the irradiation direction information indicates the direction in which the projector irradiates the component with light for each of the plurality of projectors, and the area calculation unit calculates the poorly irradiated area for each of the plurality of projectors. With this configuration, it is possible to obtain the poorly irradiated area that occurs when the component is irradiated with light from each of the plurality of projectors.

The three-dimensional measurement calculation device may further include a shape calculation unit that calculates multiple unidirectional shape data by performing, for each of the multiple projectors, a calculation to calculate unidirectional shape data indicating shape-related values, which are values related to the three-dimensional shape, for each pixel based on a pattern image acquired by using the imaging unit to capture light irradiated from the projector to the substrate, and the shape calculation unit performs a first integrated calculation process to integrate the multiple unidirectional shape data by calculating, for each pixel, the average of the shape-related values indicated by each of the multiple unidirectional shape data to calculate the three-dimensional shape of the measurement object, and the three-dimensional shape of the measurement object may be calculated in the first integrated calculation process by taking a weighted average in which the shape-related values indicated by the unidirectional shape data of pixels included in the poorly irradiated area are multiplied by a weighting coefficient that is less than 1 and greater than or equal to 0.

In this configuration, unidirectional shape data indicating values related to the three-dimensional shape (shape-related values) for each pixel is calculated based on a pattern image acquired by capturing light irradiated from a projector onto a board by an imaging unit. In other words, the unidirectional shape data is data related to the three-dimensional shape calculated based on a pattern image capturing light irradiated onto a component from one projector. The calculation for calculating this unidirectional shape data is performed for each of the multiple projectors. As a result, unidirectional shape data is acquired when light is irradiated onto a component from each of the multiple projectors. The multiple unidirectional shape data thus acquired are based on pattern images acquired while irradiating light onto a component from different directions. Therefore, a pixel corresponding to a poorly irradiated area in one unidirectional shape data may correspond to an area where light is well irradiated in another unidirectional shape data. Therefore, by calculating a weighted average of the shape-related values indicated by each of the multiple unidirectional shape data for each pixel, the multiple unidirectional shape data can be integrated to calculate the three-dimensional shape of the measurement object. Moreover, in the first integrated calculation process, the three-dimensional shape of the measurement object is calculated by a weighted average in which, among the shape-related values indicated by the one-directional shape data, the shape-related values of the pixels included in the poorly irradiated area are multiplied by a weighting coefficient less than 1 and equal to or greater than 0. This makes it possible to appropriately calculate the three-dimensional shape of the measurement object while suppressing the influence of the poorly irradiated area.

The three-dimensional measurement calculation device may be configured so that in the first integrated calculation process, the three-dimensional shape of the measurement object is calculated by a weighted average in which the shape-related values of pixels included in the poorly irradiated areas are multiplied by a weighting coefficient of 0. In this way, the operation of multiplying by a weighting coefficient of 0 is an operation of excluding the poorly irradiated areas. This makes it possible to appropriately calculate the three-dimensional shape of the measurement object while eliminating the influence of the poorly irradiated areas.

Various specific contents of the shape-related value are assumed. For example, the shape-related value may be given by the product of a calculated value of a three-dimensional shape calculated based on a pattern image and the reliability of the calculated value, or the shape-related value may be a calculated value of a three-dimensional shape calculated based on a pattern image.

The shape calculation unit may also be configured to perform both the second integrated calculation process, which integrates the multiple unidirectional shape data by calculating a representative value of the shape-related value indicated by the multiple unidirectional shape data for each pixel without weighting according to the poorly irradiated area, and the first integrated calculation process, which calculates the three-dimensional shape of the measurement object, on the same measurement object, and to perform an evaluation process for evaluating the difference in the three-dimensional shapes of the measurement object calculated by each of the first integrated calculation process and the second integrated calculation process, on a predetermined number of measurement objects from which pattern images are obtained by the three-dimensional measuring device, thereby performing a necessity determination to determine whether or not the first integrated calculation process is necessary, and after it is determined in the necessity determination that the first integrated calculation process is unnecessary, the shape calculation unit may calculate the three-dimensional shape of the measurement object by the second integrated calculation process without performing the first integrated calculation process. In this configuration, when the poorly irradiated area is not a problem, the three-dimensional shape of the measurement object can be accurately calculated by the second integrated calculation process, which is simpler than the first integrated calculation process. Here, the representative value of the shape-related value indicated by a plurality of one-way shape data items without weighting according to the poorly irradiated area includes a simple average value or a median value.

The three-dimensional measurement calculation device may be configured to further include a setting operation unit that accepts an operation to set the predetermined number. In such a configuration, the user can appropriately adjust the period during which both the first integrated calculation process and the second integrated calculation process are executed.

According to the present invention, when measuring the three-dimensional shape of a measurement object based on a pattern image obtained by capturing an image of a predetermined pattern of light irradiated onto the measurement object, it is possible to obtain poorly irradiated areas where poor light irradiation occurs.

1 is a block diagram illustrating a three-dimensional measuring apparatus according to the present invention; 2 is a plan view showing a schematic relationship between a projector and an imaging field of view included in the three-dimensional measuring apparatus shown in FIG. 1 . FIG. 2 is a block diagram showing details of a control device provided in the three-dimensional measuring apparatus of FIG. 1 . FIG. 4 is a diagram showing the content of a part model created by a part model acquisition unit. FIG. 13 is a diagram showing, in table form, an example of board configuration data used to create a part model. FIG. 2 is a diagram showing, in a table format, an example of part data used to create a part model. FIG. 11 is a diagram illustrating an example of a created part model. FIG. 4 is a diagram showing a schematic view of a poorly irradiated region calculated by a region calculation unit; 11 is a flowchart showing an example of a calculation for calculating a poorly irradiated area. 11 is a flowchart showing a first example of three-dimensional measurement. 9 is a flowchart showing omnidirectional shape data acquisition performed in the three-dimensional measurement of FIG. 8 . 9 is a flowchart showing a first data integration process executed in the three-dimensional measurement of FIG. 8 . FIG. 11 is a diagram illustrating an example of a calculation executed in the first data integration process of FIG. 10 . 8 is a flowchart showing an example of a calculation for correcting the poorly irradiated area calculated by FIG. 7 . 11 is a flowchart showing a second example of three-dimensional measurement. 14 is a flowchart showing a second data integration process executed in the three-dimensional measurement of FIG. 13 . FIG. 14 is a diagram showing a schematic diagram of the contents of calculations executed in the three-dimensional measurement of FIG. 13 .

Figure 1 is a block diagram that illustrates a three-dimensional measuring device according to the present invention. In this figure and the following figures, the horizontal X direction, the horizontal Y direction perpendicular to the X direction, and the vertical Z direction are appropriately indicated. The three-dimensional measuring device 1 in Figure 1 measures the three-dimensional shape (external shape) of a measurement object J by controlling a transport conveyor 2, a measurement head 3, and a drive mechanism 4 by a control device 100. The measurement object J is composed of a board B (printed circuit board) and a component E mounted on the board B. This measurement object J is produced by a surface mount machine that mounts the component E on the board B, and is brought into the three-dimensional measuring device 1.

The transport conveyor 2 transports the measurement object J along a predetermined transport path. Specifically, the transport conveyor 2 transports the measurement object J before measurement to a measurement position within the three-dimensional measuring device 1, and holds the measurement object J at the measurement position so that the board B is horizontal. Furthermore, when measurement of the three-dimensional shape of the measurement object J at the measurement position is completed, the transport conveyor 2 transports the measurement object J after measurement out of the three-dimensional measuring device 1.

The measurement head 3 has an imaging camera 31 that images the imaging field V31 from above, and the measurement object J brought into the measurement position is placed in the imaging field V31 and imaged by the imaging camera 31. The imaging camera 31 has a solid-state imaging element 311 that detects reflected light from the measurement object J, and captures an image of the measurement object J by the solid-state imaging element 311.

Furthermore, the measurement head 3 has a projector 32 that projects a striped pattern light L (S) whose light intensity distribution changes sinusoidally into the imaging field of view V31. The projector 32 has a light source such as an LED (Light Emitting Diode) and a digital micromirror device that reflects the light from the light source toward the imaging field of view V31. The projector 32 can project multiple types of pattern light L (S) with different phases into the imaging field of view V31 by adjusting the angle of each micromirror of the digital micromirror device. In other words, the measurement head 3 can measure the three-dimensional shape of the measurement object J in the imaging field of view V31 by the phase shift method by capturing an image with the imaging camera 31 while changing the phase of the pattern light L (S) projected from the projector 32.

2 is a plan view showing a schematic relationship between the projectors and the imaging field of view of the three-dimensional measuring device of FIG. 1. As shown in FIG. 2, the measurement head 3 has multiple (four in this example) projectors 32 (two projectors 32 are shown as representatives in FIG. 1 for the sake of simplicity). Each projector 32 projects pattern light L (S) from an obliquely upward direction onto the imaging field of view V31 of the imaging camera 31. In a plan view, the multiple projectors 32 are arranged so as to surround the periphery of the imaging camera 31, and are arranged at equal pitches in a circular shape centered on the vertical direction Z. Therefore, the multiple projectors 32 project pattern light L (S) onto the imaging field of view V31 from different projection directions D. The number of projectors 32 included in the measurement head 3 is not limited to four as shown in the example of FIG. 2.

The measurement head 3 also has an illumination 33 (FIG. 1) that irradiates light onto the imaging field of view V31. The above-mentioned projector 32 projects pattern light L(S) onto the imaging field of view V31 when measuring a three-dimensional shape, whereas the illumination 33 irradiates illumination light onto the imaging field of view V31 when capturing a two-dimensional image with the imaging camera 31.

The drive mechanism 4 supports the measurement head 3 and drives the measurement head 3 in the X, Y, and Z directions by a motor. By driving the drive mechanism 4, the measurement head 3 moves above the measurement target location of the measurement object J, and the measurement target location can be captured within the imaging field of view V31, and the three-dimensional shape of the measurement target location within the imaging field of view V31 can be measured. In particular, the three-dimensional measuring device 1 can measure the three-dimensional shape of the component E and its surroundings (measurement target location) while keeping the component E within the imaging field of view V31, which is useful for inspecting the lift of the component E from the board B, for example the lift of the terminals of a package such as a QFP (Quad Flat Package) from the board B.

The control device 100 has a main control unit 110, which is a processor composed of a CPU (Central Processing Unit) and memory, and the main control unit 110 controls each part of the device to measure the three-dimensional shape. The control device 100 also has a UI (User Interface) 200 composed of input/output devices such as a display, a keyboard, and a mouse, and a user can input commands to the control device 100 through the UI 200 and check the measurement results by the control device 100. Furthermore, the control device 100 has a projection control unit 120, an imaging control unit 130, and a drive control unit 140. The projection control unit 120 controls the projection of the pattern light L (S) by the projector 32. The imaging control unit 130 controls the imaging of the imaging field V31 by the imaging camera 31 and the irradiation of light from the lighting 33 to the imaging field V31. The drive control unit 140 controls the drive of the measurement head 3 by the drive mechanism 4.

When the conveyor 2 carries the measurement object J to the measurement position, the main control unit 110 controls the drive mechanism 4 by the drive control unit 140 to move the measurement head 3 above the measurement target location of the measurement object J. As a result, the measurement target location falls within the imaging field V31 of the imaging camera 31. Next, the main control unit 110 projects the pattern light L(S) from the projector 32 to the imaging field V31, while imaging the pattern light L(S) projected into the imaging field V31 with the imaging camera 31 (pattern imaging operation). Specifically, the control device 100 has a memory unit 150, and reads out the projection pattern T(S) stored in the memory unit 150. Then, the main control unit 110 controls the projection control unit 120 based on the projection pattern T(S) read out from the memory unit 150 to adjust the angle of each micromirror of the digital micromirror device of the projector 32 according to the projection pattern T(S). Thus, the pattern light L(S) having the projection pattern T(S) is projected onto the imaging field of view V31. Furthermore, the main control unit 110 controls the imaging control unit 130 to capture the pattern light L(S) projected onto the imaging field of view V31 with the imaging camera 31 to obtain a pattern image I(S). This pattern image I is stored in the storage unit 150. Note that the storage unit 150 stores four types of projection patterns T(S) with phases differing from each other by 90 degrees, and the pattern imaging operation is performed four times while changing the projection pattern T(S) (S=1, 2, 3, 4). As a result, four types of pattern images I(S) are obtained by capturing the pattern light L(S) with phases differing by 90 degrees. The main control unit 110 obtains the height of the imaging field of view V31 for each pixel of the imaging camera 31 from the four types of pattern images I(S) thus acquired by the phase shift method. The variations of the projection pattern T(S) are not limited to this example. For example, eight types of projection pattern T(S) with a phase difference of 45 degrees each may be used, or three types of projection pattern T(S) with a phase difference of 120 degrees each may be used.

Figure 3 is a block diagram showing details of the control device provided in the three-dimensional measuring device of Figure 1. The control device 100 stores the three-dimensional measuring program 92 downloaded from an external server computer 91 in the storage unit 150. The provision of the three-dimensional measuring program 92 is not limited to being downloaded from an external source, and the three-dimensional measuring program 92 may be provided in a state recorded on a DVD (Digital Versatile Disc) or USB (Universal Serial Bus). When the main control unit 110 executes the three-dimensional measuring program 92, a part model acquisition unit 111, an area calculation unit 112, a pattern image acquisition unit 113, and a shape calculation unit 114 are configured in the main control unit 110. These functional units 111, 112, 113, and 114 perform functions corresponding to poorly irradiated areas caused by the component E on the board B when the pattern light L (S) is projected onto the measurement object J. The details of these are as follows.

Figure 4 is a diagram showing the contents of the part model created by the part model acquisition unit, Figure 5A is a diagram showing in table form an example of board configuration data used to create the part model, Figure 5B is a diagram showing in table form an example of part data used to create the part model, and Figure 5C is a diagram showing a schematic example of a created part model. The part model acquisition unit 111 uses board configuration data 81 showing the configuration of board B and part data 82 showing the outer shape of part E to be mounted on board B to create a part model 83 showing the three-dimensional existence range of part E mounted on board B. The board configuration data 81 and part data 82 are stored in advance in the memory unit 150.

The board configuration data 81 (FIG. 5A) indicates the component mounting range Ref(R) provided on the board B and the type of component E to be mounted in the component mounting range Ref(R) for each component mounting range Ref(1), Ref(2), Ref(3), ... (R = 1, 2, 3, ...). In the board configuration data 81, the component mounting range Ref(R) is specified by the position (X coordinate) of the component mounting range Ref(R) in the X direction and the position (Y coordinate) of the component mounting range Ref(R) in the Y direction. Note that the component mounting range Ref(R) is a range that is defined by, for example, a land provided on the board B and has a spread. In contrast, the position (x, y) of the component mounting range Ref(R) corresponds to the center of the component mounting range Ref(R) in a planar view. The component mounting range Ref(R) indicated by the board configuration data 81 is an ideal range in which the component E should be mounted, and is not a range in which the component E is actually mounted by a surface mounter. As the board configuration data 81, CAD data showing the configuration of the board B, inspection data showing standards for inspecting whether the positional relationship between the component E mounted in the component mounting range Ref(R) by the surface mounter and the component mounting range Ref(R) is appropriate, or the like can be used.

5B, the part data 82 indicates the configuration of part E for each type of part E. Specifically, the part data 82 indicates the outer shape of part E, such as the length El, width Ew, and height Eh of rectangular part E, and the presence or absence of the property of reflecting light (pattern light L(S)) (presence or absence of reflection), for each of the various parts Ea, Eb, Ec, ....

The component model acquisition unit 111 checks the type of component E to be mounted in the component mounting range Ref(R) of the board B using the board configuration data 81, and creates a component model 83 based on the result of checking the configuration of the corresponding type of component E using the component data 82. This component model 83 indicates the position (x, y) of the component mounting range Ref(R) and the outer shape (the component's X size E_x, Y size E_y, and Z size E_z) of the component E to be mounted in the component mounting range Ref(R). In other words, the component model 83 indicates the range in which the component E mounted in the component mounting range Ref(R) protrudes in the Z direction (height direction) from the board B. Furthermore, the component model 83 indicates whether the component E mounted in the component mounting range Ref(R) has the property of reflecting light (pattern light L(S)). This component model 83 is created for each of the multiple component mounting ranges Ref(R) provided on the board B.

Incidentally, the specific data used to create the part model 83 and the specific method of creating the part model 83 using each piece of data are not limited to the examples given here. For example, when the part model 83 is input by a user's operation on the UI 200, the part model acquisition unit 111 does not need to create the part model 83 based on the board configuration data 81 and the part data 82, but only needs to acquire the part model 83 input by the user.

Figure 6 is a diagram showing a schematic of the contents of the poorly irradiated area calculated by the area calculation unit. The component E shown in Figure 6 has the property of reflecting the pattern light L(S). In Figure 6, the component E mounted in the component mounting range Ref(R) has a width E_x in the X direction, a length E_y in the Y direction, and a height E_z in the Z direction. The pattern light L(S) emitted from the projector 32 is projected onto the component E from the projection direction D.

As a result, in a plan view, a shadow area As (poorly irradiated area) occurs downstream of the part E in the projection direction D. This shadow area As is an area where the pattern light L (S) projected from the projector 32 onto the part E is blocked by the part E, causing a shadow of the part E on the board B. In addition, in a plan view, a secondary reflection area Ar (poorly irradiated area) occurs upstream of the part E in the projection direction D. This secondary reflection area Ar is an area where the light projected from the projector 32 onto the part E and reflected by the side of the part E enters the board B. In this example, the length of the shadow area As and the secondary reflection area Ar in the Y direction corresponds to the length E_y of the part E in the Y direction. In addition, the width of the shadow area As and the secondary reflection area Ar in the X direction corresponds to the width obtained by multiplying the height E_z of the part E by the tangent (tan θs) of the projection elevation angle θs. Note that the secondary reflection area Ar does not occur for parts E that do not reflect the pattern light L (S).

Such poorly illuminated areas (shadow areas As and secondary reflection areas Ar) are calculated for each of the multiple projectors 32. Next, this point will be explained using FIG. 7. FIG. 7 is a flowchart showing an example of a calculation for calculating the poorly illuminated areas. The flowchart in FIG. 7 is executed by the calculation of the main control unit 110.

In step S101, a count value P (= 1, 2, 3, 4) identifying the projector 32 is reset to zero, and in step S102, the count value P is incremented. Furthermore, in step S103, a count value R (= 1, 2, 3, ...) identifying the component mounting range Ref is reset to zero, and in step S104, the count value R is incremented.

In step S105, the part model acquisition unit 111 creates a part model 83 of part E to be mounted in the Rth part mounting range Ref(R). In step S106, the area calculation unit 112 confirms the projection direction D in which the Pth projector 32 projects the pattern light L(S) onto the Rth part mounting range Ref. Specifically, projection direction information 84 indicating the projection direction D of the pattern light L(S) by the projector 32 for each of the multiple projectors 32 is stored in advance in the storage unit 150, and the area calculation unit 112 confirms the projection direction D by referring to the projection direction information 84.

In step S107, the area calculation unit 112 calculates a shadow area As (reflection poor area) that occurs when the pattern light L(S) is projected from the projection direction D by the Pth projector 32 onto the part E represented by the part model 83 created for the Rth part mounting range Ref. Furthermore, if the part E represented by the part model 83 created for the Rth part mounting range Ref reflects the pattern light L(S), the area calculation unit 112 calculates a secondary reflection area Ar (reflection poor area) that occurs when the pattern light L(S) is projected from the projection direction D by the Pth projector 32 onto the part E.

In step S108, it is confirmed whether the count value R of the component mounting range Ref has reached the number Rmax of component mounting ranges Ref provided on the board B, that is, whether steps S105 to S107 have been performed for all component mounting ranges Ref provided on the board B. If there is a component mounting range Ref for which steps S105 to S107 have not been performed (step S108: "NO"), the process returns to step S104. In this way, steps S104 to S107 are repeated until steps S105 to S107 have been performed for all component mounting ranges Ref (step S108: "YES"). As a result, the poorly irradiated areas (shadow areas As and secondary reflection areas Ar) that occur when the pattern light L (S) is projected from the Pth projector 32 are calculated for the components E in all component mounting ranges Ref.

In step S109, it is confirmed whether the count value P of the projector 32 has reached the number Pmax (=4) of the projectors 32, that is, whether steps S103 to S108 have been executed for all the projectors 32. If there is a projector 32 for which steps S103 to S108 have not been executed (if "NO" in step S109), the process returns to step S102. In this way, steps S102 to S108 are repeated until steps S103 to S108 are executed for all the projectors 32 (if "YES" in step S109). As a result, when the pattern light L (S) is projected from the projector 32, poorly irradiated area information 85 indicating the poorly irradiated areas (shadow areas As and secondary reflection areas Ar) caused by the components E in each component mounting range Ref for all the projectors 32 is calculated and stored in the memory unit 150.

Using the poorly irradiated area information 85 thus acquired, three-dimensional measurement of the measurement object J is performed (Figure 8). Figure 8 is a flowchart showing a first example of three-dimensional measurement, Figure 9 is a flowchart showing the acquisition of all-directional shape data performed in the three-dimensional measurement of Figure 8, Figure 10 is a flowchart showing the first data integration process performed in the three-dimensional measurement of Figure 8, and Figure 11 is a diagram showing the calculation performed in the first data integration process of Figure 10. The flowcharts of Figures 8, 9, and 10 are executed under the control of the main control unit 110.

In the three-dimensional measurement of Fig. 8, omnidirectional shape data acquisition is performed in step S201. As shown in Fig. 9, in omnidirectional shape data acquisition, recognition of the fiducial mark attached to the board B brought into the three-dimensional measuring device 1 is performed (step S301). Specifically, with the imaging camera 31 facing the fiducial mark in the imaging field of view V31 from above, the imaging camera 31 captures the fiducial mark while irradiating the fiducial mark with illumination light from the illumination 33 to obtain a mark image. The mark image is sent from the imaging control unit 130 to the main control unit 110, and the main control unit 110 detects the position of the board B held on the transport conveyor 2 from the position of the fiducial mark appearing in the mark image.

In step S302, the count value P of the projector 32 is reset to zero, and in step S303, the count value P is incremented. Furthermore, in step S304, the count value R of the component mounting range Ref is reset to zero, and in step S305, the count value R is incremented.

When starting step S306, the main control unit 110 adjusts the position of the board B by the drive mechanism 4 so that the Rth component mounting range Ref falls within the imaging field of view V31. In particular, the main control unit 110 performs position adjustment of the board B to the imaging field of view V31 based on the position of the board B detected in step S301. Then, in step S306, the pattern image acquisition unit 113 controls the projection control unit 120 and the imaging control unit 130 to project the pattern light L(S) from the Pth projector 32 onto the component E in the Rth component mounting range Ref, while capturing the pattern light L(S) with the imaging camera 31. As a result, as described above, four types of pattern images I(S) corresponding to different phases are acquired (S=1, 2, 3, 4) and stored in the memory unit 150.

In step S307, the shape calculation unit 114 calculates the unidirectional shape data 86 based on the four types of pattern images I(S) by the phase shift method. This unidirectional shape data 86 is data indicating the shape related value Q related to the three-dimensional shape of the component E mounted in the component mounting range Ref for each pixel PX (FIG. 11). Here, the pixel PX corresponds to, for example, a pixel of the solid-state imaging element 311, and since the imaging camera 31 captures an image while facing the board B from the Z direction, the multiple pixels of the solid-state imaging element 311 correspond to different positions (X coordinate, Y coordinate) on the board B. This shape related value Q is composed of a measurement value Qm calculated as a height (Z coordinate) from the four types of pattern images I(S) by the phase shift method and a reliability Qr of the measurement value Qm. In particular, in this example, the shape related value Q is given by the product of the measurement value Qm and the reliability Qr.

Incidentally, in the phase shift method, the reliability Qr of each pixel PX can be calculated based on the difference in luminance of the pixel PX of four types of pattern images I(S) (S=1, 2, 3, 4). Specifically, if the luminance values of the four types of pattern images I(S) captured while shifting the phase by π/2 are d0, d1, d2, and d3, the phase shift angle α is expressed by the following formula: α=a tan[(d2-d0)/(d3-d1)]
The reliability Qr is calculated by the following formula: Qr=[(d2-d0) ² +(d3-d1) ² ] ^1/2
It is calculated as follows.

Note that the pixel PX used when calculating the shape-related value Q does not have to correspond to a pixel of the solid-state imaging element 311. For example, if image processing is performed to convert the resolution of an image captured by the solid-state imaging element 311, the shape-related value Q may be calculated using the pixel PX that corresponds to the converted resolution.

In step S308, it is confirmed whether the count value R of the component mounting range Ref has reached the number Rmax of component mounting ranges Ref provided on board B, that is, whether the acquisition of pattern images I(S) (step S306) and the calculation of unidirectional shape data 86 (step S307) have been performed for all component mounting ranges Ref provided on board B. If there is a component mounting range Ref for which steps S306 to S307 have not been performed ("NO" in step S308), the process returns to step S305. In this way, steps S306 to S307 are repeated until the acquisition of pattern images I(S) (step S306) and the calculation of unidirectional shape data 86 (step S307) have been performed for all component mounting ranges Ref ("YES" in step S308). As a result, the one-way shape data 86 in the case where the pattern light L(S) is projected from the P-th projector 32 is calculated for the components E in all component mounting ranges Ref.

In step S309, it is confirmed whether the count value P of the projectors 32 has reached the number Pmax (=4) of the projectors 32, i.e., whether steps S304 to S308 have been executed for all the projectors 32. If there is a projector 32 for which steps S304 to S308 have not been executed (step S309: "NO"), the process returns to step S303. In this way, steps S304 to S308 are repeated until steps S304 to S308 have been executed for all the projectors 32 (step S309: "YES"). As a result, the unidirectional shape data 86 calculated for the component E in each component mounting range Ref when the pattern light L(S) is projected from the projector 32 is acquired for all the projectors 32 and stored in the memory unit 150.

Thus, when the omnidirectional shape data acquisition of the three-dimensional measurement in FIG. 9 (step S201 in FIG. 8) is completed, the first data integration process (step S202) is executed. In the first data integration process shown in FIG. 10, the count value R (=1, 2, 3, ...) that identifies the component mounting range Ref is reset to zero (step S401), and the count value R is incremented (step S402). As a result, four unidirectional shape data 86 acquired by projecting the projection pattern T (S) from each of the four projectors 32 are specified for the component E mounted in the Rth component mounting range Ref. In other words, four unidirectional shape data 86 corresponding to the four projectors 32 are specified for the component E in the Rth component mounting range Ref.

Next, the count value N (N=1, 2, 3, ...) identifying the pixel PX is reset to zero (step S403), and the count value N is incremented (step S404). Furthermore, in step S405, the count value P (=1, 2, 3, 4) identifying the projector 32 is reset to zero, and in step S406, the count value P is incremented. This specifies the Nth pixel PX of the unidirectional shape data 86 corresponding to the Pth projector 32.

Then, the shape calculation unit 114 determines whether the Nth pixel PX thus specified belongs to the shadow area As or the secondary reflection area Ar based on the poor irradiation area information 85 in the storage unit 150 (step S407), and determines the weighting factor W(P) to be multiplied by the shape-related value Q (=measurement value Qm x reliability Qr) of the Nth pixel PX (steps S408, S409). In other words, if it is determined based on the poor irradiation area information 85 that the Nth pixel PX belongs to the shadow area As or the secondary reflection area Ar ("YES" in step S407), the weighting factor W(P) for the shape-related value Q of the Nth pixel PX is determined to be "0", and the shape-related value Q is excluded (step S408). On the other hand, when the poorly irradiated area information 85 determines that the Nth pixel PX does not belong to either the shadow area As or the secondary reflection area Ar ("NO" in step S407), the weighting coefficient W(P) for the shape-related value Q of the Nth pixel PX is determined to be 1 (step S410).

The calculations of steps S407 to S409 for the shape-related value Q of the Nth pixel PX are repeated while incrementing the count value P of the projector 32 (step S406) until the count value P reaches Pmax (=4) ("YES" in step S410). This determines the weighting coefficient W(P) for the shape-related value Q of the Nth pixel PX corresponding to each of P=1, 2, 3, and 4. In the example of FIG. 11, the Nth pixel PX corresponding to the projector 32 with P=2 is determined to belong to the shadow area As or the secondary reflection area Ar, and the weighting coefficient W(2) for the shape-related value Q of the pixel PX is determined to be "0". On the other hand, the Nth pixel PX corresponding to the projector 32 with P=1, 3, and 4 is determined to belong to neither the shadow area As nor the secondary reflection area Ar, and the weighting coefficients W(1), W(3), and W(4) for the shape-related value Q of these pixels PX are determined to be "1".

In step S411, the shape calculation unit 114 performs a calculation to obtain a weighted average of the shape-related values Q of the Nth pixel PX corresponding to each projector 32 where P = 1, 2, 3, 4 using the weighting coefficient W(P) determined in steps S407 to S409, thereby obtaining an integrated value H that integrates these shape-related values Q. The weighted average formula for obtaining the integrated value H is as shown in the "Conversion formula" column in Figure 11. As a result, in the example of Figure 11, the integrated value H of the Nth pixel PX is 159.

The calculations of steps S405 to S411 are repeated while incrementing the count value N (step S404) until the count value N reaches the number Nmax of pixels PX constituting the unidirectional shape data 86 ("YES" in step S412). In this way, integrated shape data 87 indicating an integrated value H for each pixel PX obtained by integrating four shape-related values Q corresponding to different projectors 32 is calculated and stored in the memory unit 150. Such integrated shape data 87 corresponds to data obtained by integrating four unidirectional shape data 86 corresponding to four projectors 32 obtained for a component E mounted in the Rth component mounting range Ref, and indicates the three-dimensional shape of the E.

The calculations of steps S403 to S412 are repeated while incrementing the count value R (step S402) until the count value R reaches the number Rmax of component mounting ranges Ref ("YES" in step S413). In this way, integrated shape data 87 is calculated for components E in all component mounting ranges Ref.

In the embodiment described above, a component model 83 (reference model) showing the component mounting range Ref on the board B where the component E is mounted and the outer shape of the component E mounted in the component mounting range Ref is acquired (step S105). Then, based on the projection direction information 84 (irradiation direction information) showing the projection direction D in which the pattern light L (S) is projected onto the component E by the three-dimensional measuring device 1 and the component model 83, the shadow area As and the secondary reflection area Ar (irradiation failure area) are calculated (step S107). In this way, when measuring the three-dimensional shape of the measurement object J based on the pattern image I (S) acquired by capturing the pattern light L (S) irradiated onto the measurement object J, it is possible to acquire the shadow area As and the secondary reflection area Ar where irradiation failure of the pattern light L occurs.

The component model acquisition unit 111 (reference model acquisition unit) creates a component model 83 from at least one of the inspection data indicating the standard for inspecting the suitability of the positional relationship between the component E mounted in the component mounting range Ref and the component mounting range Ref and the CAD data indicating the configuration of the board (step S105). In this configuration, the component model 83 is created using existing data obtained from the inspection data or CAD data of the board B, and the shadow area As and secondary reflection area Ar can be calculated based on this component model 83.

In addition, a plurality of projectors 32 (pattern irradiation units) are provided that irradiate the part E with pattern light L (S) from different projection directions D, and the projection direction information 84 indicates the projection direction D in which light is irradiated from the projector 32 to the part E for each of the plurality of projectors 32. In this configuration, if the projector 32 that projects the pattern light L (S) changes among the plurality of projectors 32, the shadow area As and the secondary reflection area Ar also change. In response to this, the area calculation unit 112 calculates the shadow area As and the secondary reflection area Ar for each of the plurality of projectors 32 (steps S102, S107). This makes it possible to obtain the shadow area As and the secondary reflection area Ar that are generated when light is projected from each of the plurality of projectors 32 onto the part E.

Based on the pattern image I(S) acquired by capturing the pattern light L(S) irradiated from the projector 32 onto the substrate B by the imaging camera 31 (imaging unit), the shape calculation unit 114 performs a calculation (step S307) to calculate unidirectional shape data 86 indicating a shape related value Q, which is a value related to a three-dimensional shape, for each pixel PX. This calculation (step S307) is performed for each of the multiple projectors 32 (step S303), and multiple unidirectional shape data 86 corresponding to different projectors 32 are calculated. Furthermore, the shape calculation unit 114 performs a first integration calculation process (FIG. 10) to integrate the multiple unidirectional shape data 86 by calculating a weighted average of the shape related value Q indicated by each of the multiple unidirectional shape data 86 for each pixel PX, thereby calculating the three-dimensional shape of the measurement object J. In this first integrated calculation process, the three-dimensional shape of the measurement object J is calculated by taking a weighted average (step S411) in which the shape-related values Q of pixels PX included in the shadow region As or secondary reflection region Ar, among the shape-related values Q indicated by the unidirectional shape data 86, are multiplied by a weighting coefficient W(P) that is less than 1 and is equal to or greater than 0.

In this configuration, the unidirectional shape data 86 indicating the shape related value Q for each pixel PX regarding the three-dimensional shape is calculated based on the pattern image I(S) acquired by capturing the pattern light L(S) irradiated from the projector 32 onto the board B by the imaging camera 31 (step S307). In other words, the unidirectional shape data 86 is data regarding the three-dimensional shape calculated based on the pattern image I(S) capturing the pattern light L(S) irradiated onto the part E from one projector 32. The calculation (step S307) for calculating the unidirectional shape data 86 is executed for each of the multiple projectors 32 (step S303). As a result, the unidirectional shape data 86 is acquired when the pattern light L(S) is irradiated onto the part E from each of the multiple projectors 32. The multiple unidirectional shape data 86 thus acquired are based on the pattern image I(S) acquired while irradiating the part E with the pattern light L(S) from each of the multiple projectors 32. Therefore, a pixel PX corresponding to a shadow region As or a secondary reflection region Ar in one unidirectional shape data 86 may correspond to a region well irradiated with light in another unidirectional shape data 86. Therefore, by calculating an average of the shape-related values Q indicated by each of the plurality of unidirectional shape data 86 for each pixel PX, the plurality of unidirectional shape data 86 can be integrated to calculate the three-dimensional shape of the measurement object J (FIG. 10). Moreover, in the first integration calculation process in FIG. 10, the three-dimensional shape of the measurement object J is calculated by multiplying the shape-related values Q of the pixels PX included in the shadow region As or the secondary reflection region Ar, among the shape-related values Q indicated by the unidirectional shape data 86, by a weighting coefficient W(P) less than 1 and equal to or greater than 0 (steps S407 to S410). This makes it possible to appropriately calculate the three-dimensional shape of the measurement object J while suppressing the influence of the shadow region As or the secondary reflection region Ar.

10, the three-dimensional shape of the measurement object J is calculated by a weighted average in which the shape-related values Q of pixels PX contained in the shadow region As or secondary reflection region Ar are multiplied by a weighting coefficient W(P) of 0 (steps S407 to S410). In this way, the operation of multiplying by a weighting coefficient W(P) of 0 is an operation of excluding the shadow region As and the secondary reflection region Ar. This makes it possible to appropriately calculate the three-dimensional shape of the measurement object J while eliminating the influence of the shadow region As and the secondary reflection region Ar.

Figure 12 is a flowchart showing an example of a calculation for correcting the poorly irradiated area calculated by Figure 7. The flowchart in Figure 12 is executed by the calculation of the main control unit 110 in parallel with the acquisition of the omnidirectional shape data in Figure 9. In step S501, the count value P of the projector 32 is reset to zero, and in step S502, the count value P is incremented. In addition, in step S503, the count value R of the component mounting range Ref is reset to zero, and in step S504, the count value R is incremented. As a result, the shadow area As and secondary reflection area Ar calculated for the component E of the Rth component mounting range Ref are specified in correspondence with the Pth projector 32.

In step S505, the shadow area As and secondary reflection area Ar specified in steps S502 and S504 are corrected based on the position of the substrate B detected in step S301 of the omnidirectional shape data acquisition in Figure 9. In other words, it is assumed that the position of the substrate B detected in step S301 is shifted by a positional deviation amount Δa from the ideal position of the substrate B assumed in the calculation of the shadow area As and secondary reflection area Ar. In such a case, by correcting the shadow area As and secondary reflection area Ar according to the positional deviation amount Δa of the substrate B, the shadow area As and secondary reflection area Ar can be accurately calculated according to the actual position of the substrate B brought into the three-dimensional measurement device 1.

In step S506, the shadow area As and secondary reflection area Ar specified in steps S502 and S504 are corrected based on the position of the component E actually mounted in the Rth component mounting range Ref. The details of this are as follows.

In the control using the poorly irradiated area correction of Fig. 12, in step S306 of acquiring omnidirectional shape data of Fig. 9, a two-dimensional image of component E is acquired in addition to the pattern image I(S). Specifically, the illumination 33 irradiates illumination light onto component E within the imaging field of view V31 while the imaging camera 31 images component E from above, thereby acquiring a two-dimensional image of component E. Furthermore, in step S306, the component position indicating the positional relationship between component E actually mounted in the component mounting range Ref of the board B brought into the three-dimensional measuring device 1 and the component mounting range Ref is calculated based on the two-dimensional image of component E, and is stored in the memory unit 150.

In contrast, it is assumed that the positional relationship between the component mounting range Ref indicated by the component position calculated in step S306 and the component E deviates by a positional deviation amount Δb from the ideal positional relationship assumed in the calculation of the shadow area As and the secondary reflection area Ar. In such a case, by correcting the shadow area As and the secondary reflection area Ar according to the positional deviation amount Δb of the component E relative to the component mounting range Ref, the shadow area As and the secondary reflection area Ar can be accurately calculated according to the actual position of the component E mounted in the component mounting range Ref of the board B brought into the three-dimensional measurement device 1.

In step S507, it is confirmed whether the count value R of the component mounting range Ref has reached the number Rmax of component mounting ranges Ref provided on the board B, that is, whether the correction of steps S505 to S506 has been performed for all component mounting ranges Ref provided on the board B. If there is a component mounting range Ref for which the correction of steps S505 to S506 has not been performed (step S507: "NO"), the process returns to step S504. In this way, steps S505 to S506 are repeated until the correction of steps S505 to S506 is performed for all component mounting ranges Ref (step S507: "YES"). This corrects the poorly irradiated areas (shadow areas As and secondary reflection areas Ar) for each component mounting range Ref that occur when the pattern light L(S) is projected from the Pth projector 32.

In step S508, it is confirmed whether the count value P of the projectors 32 has reached the number Pmax (=4) of the projectors 32, i.e., whether steps S503 to S507 have been executed for all the projectors 32. If there is a projector 32 for which steps S503 to S507 have not been executed ("NO" in step S508), the process returns to step S502. In this way, steps S503 to S507 are repeated until steps S503 to S507 have been executed for all the projectors 32 ("YES" in step S508). This corrects the poorly irradiated areas (shadow areas As and secondary reflection areas Ar) that occur due to the components E in each component mounting range Ref when the pattern light L(S) is projected from each projector 32.

10, steps S407 to S411 are executed based on the shadow area As and secondary reflection area Ar thus corrected. This makes it possible to calculate integrated shape data 87 based on the shadow area As and secondary reflection area Ar that are accurately set for the measurement object J (board B and component E) that has actually been loaded into the three-dimensional measurement device 1.

12, the area calculation unit 112 calculates the shadow area As and the secondary reflection area Ar based on the result of recognizing in step S301 the position of the substrate B that has been brought into the three-dimensional measuring device 1 and supported by the transport conveyor 2 (object support unit) (step S505). In this configuration, the shadow area As and the secondary reflection area Ar can be accurately calculated according to the actual position of the substrate B that has been brought into the three-dimensional measuring device 1.

In particular, the position of the substrate B supported by the transport conveyor 2 is recognized based on the image of the fiducial mark attached to the substrate B captured by the imaging camera 31 (step S301). In this configuration, the shadow area As and the secondary reflection area Ar can be accurately calculated according to the position of the substrate B brought into the three-dimensional measurement device 1.

In addition, the area calculation unit 112 calculates the shadow area As and the secondary reflection area Ar based on the result of detecting the position of the component E mounted in the component mounting range Ref in step S306 (step S506). In this configuration, the shadow area As and the secondary reflection area Ar can be accurately calculated according to the actual position of the component E mounted in the component mounting range Ref.

In addition, the position of the component E mounted in the component mounting range Ref is detected based on a two-dimensional image of the component E (step S306). In this configuration, the shadow area As and the secondary reflection area Ar can be accurately calculated according to the actual position of the component E mounted in the component mounting range Ref.

Figure 13 is a flowchart showing a second example of three-dimensional measurement, Figure 14 is a flowchart showing a second data integration process executed in the three-dimensional measurement of Figure 13, and Figure 15 is a diagram showing a schematic diagram of the content of the calculation executed in the three-dimensional measurement of Figure 13. Here, a scene is assumed in which multiple substrates B are loaded into the three-dimensional measuring device 1 in sequence, and three-dimensional measurement is executed for each substrate B.

As shown in Figure 13, in the second example of three-dimensional measurement, omnidirectional shape data acquisition (step S601) and a first data integration process (step S602) are executed as in the first example. Furthermore, in the second example, a second data integration process (Figure 14) is executed in step S603. Note that the order of execution of the first data integration process and the second data integration process is not limited to the example of Figure 13, and may be reversed from the example of Figure 13.

As shown in Figure 14, the second data integration process differs from the first data integration process in that instead of steps S405 to S411 of the first data integration process, a simple average (an average with a total weighting coefficient of 1) is calculated in step S414. That is, in step S414 of the second data integration process, a simple average of the shape-related values Q of the Nth pixel PX obtained for each of the four projectors 32 is calculated. In this way, the integrated value H is calculated by the simple average rather than the weighted average, and the integrated shape data 87 is obtained.

Once the three-dimensional measurement of one substrate B is completed in this manner (steps S601 to S603), it is determined whether steps S601 to S603 have been performed on a predetermined number of substrates B (one or more) (step S604). If they have not been performed (NO in step S604), it is determined in step S605 whether to end the measurement. If the measurement is to be ended (YES in step S605), the three-dimensional measurement in FIG. 13 ends, whereas if the measurement is not to be ended (NO in step S605), the process returns to step S601.

In step S604, if it is determined that steps S601 to S603 have been performed for a predetermined number of boards B (YES), the difference between the integrated shape data 87 (three-dimensional shape) acquired in the first data integration process in step S602 and the integrated shape data 87 (three-dimensional shape) acquired in the second data integration process in step S603 is calculated. For example, for the same component mounting range Ref of the same board B, the average value of the squares of the differences between the integrated shape data 87 (three-dimensional shape) acquired in the first data integration process and the integrated shape data 87 (three-dimensional shape) acquired in the second data integration process (i.e., the mean square error) is calculated. Note that, if one board B has multiple component mounting ranges Ref, the mean square error may be calculated for each component mounting range Ref, or the mean square error may be calculated for one representative component mounting range Ref. Furthermore, the mean square error is calculated for each of the predetermined number of boards B. The average, median or maximum value of the mean squared errors thus calculated is calculated as the difference. Note that the specific method of calculating the difference is not limited to this example, and any calculation method that can evaluate the difference between two data can be used.

In step S607, whether or not the first data integration process is required is determined based on the difference. Specifically, if the difference is equal to or greater than a predetermined threshold, it is determined that the first data integration process is required (YES), and if the difference is less than the threshold, it is determined that the first data integration process is not required. For example, as shown in each waveform (three-dimensional shape) in the "affected by poor irradiation" column in FIG. 15, when there is an effect of poor irradiation, the peak noise that occurs in the three-dimensional shape obtained by the second data integration process does not appear in the three-dimensional shape obtained by the first data integration process. This indicates that the first data integration process is necessary to remove the effect of poor irradiation. On the other hand,
As shown in the waveforms (three-dimensional shapes) in the column "No influence of poor irradiation" in Fig. 15, when there is no influence of poor irradiation, there is no significant difference between the three-dimensional shapes obtained by the first data integration process and the three-dimensional shapes obtained by the second data integration process. This indicates that the first data integration process is unnecessary.

When it is determined that the first data integration process is necessary ("YES" in step S607), all-directional shape data acquisition (step S608), the first data integration process (step S609), and a measurement end determination (step S610) are performed each time one board B is loaded into the three-dimensional measuring device 1. In this way, the integrated shape data 87 is calculated by the first data integration process, not the second data integration process (step S609). On the other hand, when it is determined that the first data integration process is unnecessary ("NO" in step S607), all-directional shape data acquisition (step S611), the second data integration process (step S612), and a measurement end determination (step S613) are performed each time one board B is loaded into the three-dimensional measuring device 1. In this way, the integrated shape data 87 is calculated by the second data integration process, not the first data integration process (step S612).

In the example of FIG. 13, the shape calculation unit 114 executes both the first data integration process (step S602) and the second data integration process (step S603) for the same measurement object J. In this second data integration process, the shape calculation unit 114 integrates the multiple unidirectional shape data 86 by calculating the average (simple average) of the shape related value Q indicated by the multiple unidirectional shape data 86 for each pixel PX without weighting according to the shadow area As and the secondary reflection area Ar, and calculates the three-dimensional shape (integrated shape data 87) of the measurement object J. Furthermore, the shape calculation unit 114 executes an evaluation process (step S606) for evaluating the difference between the integrated shape data 87 (three-dimensional shape) of the measurement object J calculated in each of the first data integration process (step S602) and the second data integration process (step S603) for a predetermined number of measurement objects J from which the pattern image I (S) was acquired by the three-dimensional measuring device 1. Based on this evaluation process, a necessity determination is performed to determine whether or not the first data integration process is necessary (step S607). After it is determined in the necessity determination of step S607 that the first data integration process is unnecessary, the shape calculation unit 114 calculates integrated shape data 87 (three-dimensional shape) of the measurement object J by the second data integration process without executing the first data integration process (steps S611 to S613). In this configuration, when the shadow area As and the secondary reflection area Ar are not a problem, the integrated shape data 87 of the measurement object J can be accurately calculated by the second data integration process, which is simpler than the first data integration process.

In this example, the "predetermined number" determined in step S604 may be configured to be set by the user. In this case, the user executes an operation to set the predetermined number in UI200, and UI200 accepts the predetermined number input by the user and sets it to be used in the three-dimensional measurement of FIG. 13. In this configuration, the user can appropriately adjust the period (steps S601 to S604) during which both the first data integration process (step S602) and the second data integration process (step S603) are executed.

In this manner, in the above embodiment, the three-dimensional measuring device 1 corresponds to an example of the "three-dimensional measuring device" of the present invention, the control device 100 corresponds to an example of the "three-dimensional measuring calculation device" of the present invention, the part model acquisition unit 111 corresponds to an example of the "reference model acquisition unit" of the present invention, the area calculation unit 112 corresponds to an example of the "area calculation unit" of the present invention, the shape calculation unit 114 corresponds to an example of the "shape calculation unit" of the present invention, the transport conveyor 2 corresponds to an example of the "object support unit" of the present invention, the imaging camera 31 corresponds to an example of the "imaging unit" of the present invention, and the projector 3 corresponds to an example of the "object support unit" of the present invention. 2 corresponds to an example of a "projector" of the present invention, the multiple projectors 32 correspond to an example of a "pattern irradiation unit" of the present invention, the part model 83 corresponds to an example of a "reference model" of the present invention, the projection direction information 84 corresponds to an example of "irradiation direction information" of the present invention, the unidirectional shape data 86 corresponds to an example of "unidirectional shape data" of the present invention, the server computer 91 corresponds to an example of a "recording medium" of the present invention, the three-dimensional measurement program 92 corresponds to an example of a "program for three-dimensional measurement" of the present invention, and the UI 200 corresponds to a "setting operation unit" of the present invention. ", the secondary reflection area Ar corresponds to an example of a "secondary reflection area" and a "poorly irradiated area" of the present invention, the shadow area As corresponds to an example of a "shadow area" and a "poorly irradiated area" of the present invention, the substrate B corresponds to an example of a "substrate" of the present invention, the component E corresponds to an example of a "component" of the present invention, the pattern image I(S) corresponds to an example of a "pattern image" of the present invention, the measurement object J corresponds to an example of a "measurement object" of the present invention, the pattern light L(S) corresponds to an example of "light" of the present invention, the pixel PX corresponds to an example of a "pixel" of the present invention, and the shape The shape-related value Q corresponds to an example of the "shape-related value" of the present invention, the measurement value Qm corresponds to an example of the "calculated value" of the present invention, the reliability Qr corresponds to an example of the "reliability" of the present invention, the component mounting range Ref corresponds to an example of the "component mounting range" of the present invention, step S606 corresponds to an example of the "evaluation process" of the present invention, step S607 corresponds to an example of the "necessity determination" of the present invention, the first data integration process of FIG. 10 corresponds to an example of the "first integrated calculation process" of the present invention, and the second data integration process of FIG. 14 corresponds to an example of the "second integrated calculation process" of the present invention.

The present invention is not limited to the above embodiment, and various modifications can be made to the above without departing from the spirit of the invention. For example, it is not necessary to calculate both the secondary reflection area Ar and the shadow area As as a poorly irradiated area. For example, if the influence of one of the secondary reflection area Ar and the shadow area As is large and the influence of the other is small, only one of the secondary reflection area Ar and the shadow area As may be calculated as a poorly irradiated area in step S107 of FIG. 7.

In addition, in the poorly irradiated area correction of FIG. 12, it is possible to perform only one of the correction based on the position of substrate B (step S505) and the correction based on the position of component E (step S506) without performing the other.

In addition, the specific content of the shape-related value Q may be changed as appropriate. For example, the measurement value Qm may be calculated as the shape-related value Q without calculating the reliability Qr.

In step S408, the weighting coefficient W(P) by which the poorly irradiated area is multiplied does not have to be 0, but may be any value greater than or equal to 0 and less than 1.

Furthermore, it is not necessarily necessary to configure each of the functional units 111, 112, 113, and 114 configured in the main control unit 110 by executing the three-dimensional measurement program 92 in the control device 100 provided in the three-dimensional measurement device 1. Therefore, each of the functional units 111, 112, 113, and 114 may be configured in a processor provided in a computer provided separately from the three-dimensional measurement device 1.

Furthermore, in the second data integration process of FIG. 14 , in step S414, instead of calculating the average value (representative value) by simple averaging, a median value (representative value) may be calculated.

1... Three-dimensional measuring device 100... Control device (arithmetic device for three-dimensional measurement)
111: Part model acquisition unit (reference model acquisition unit)
112...Area calculation unit (area calculation unit)
114: Shape calculation unit 2: Transport conveyor (object support unit)
31...imaging camera (imaging unit)
32...Projector (pattern irradiation unit)
83... Part model (reference model)
84...Projection direction information (irradiation direction information)
86... Unidirectional shape data 91... Server computer (recording medium)
92... Three-dimensional measurement program 200... UI (setting operation unit)
Ar…Secondary reflection area (poor irradiation area)
As...Shadow area (poor irradiation area)
B... Board E... Component I(S)... Pattern image J... Measurement object L(S)... Pattern light (light)
PX: pixel Q: shape-related value Qm: measurement value (calculated value)
Qr: Reliability Ref: Part mounting range

Claims (17)

A three-dimensional measurement calculation device includes an object support section that supports a measurement object having a board and components mounted on the board, a pattern irradiation section that irradiates light of a predetermined pattern onto the measurement object supported by the object support section, and an imaging section that acquires a two-dimensional pattern image by imaging the light irradiated onto the measurement object from the pattern irradiation section, the three-dimensional measurement calculation device executing calculations for measuring a three-dimensional shape of the measurement object based on the pattern image acquired by a three-dimensional measurement device, the three-dimensional measurement device comprising:
a reference model acquisition unit that acquires a reference model indicating a component mounting area on the board on which the component is to be mounted and an outer shape of the component to be mounted in the component mounting area ;
a region calculation unit that calculates a poorly irradiated region, which is a secondary reflection region where light irradiated from the pattern irradiation unit and reflected by the component is incident on the board, based on irradiation direction information indicating a direction in which the light is irradiated onto the component by the three-dimensional measuring device and on the reference model;
the pattern irradiation unit has a plurality of projectors that irradiate the component with light from different directions;
the irradiation direction information indicates, for each of the plurality of projectors, a direction in which light is irradiated from the projector to the component;
the region calculation unit calculates the poor illumination region for each of the plurality of projectors;
a shape calculation unit that calculates a plurality of unidirectional shape data by executing, for each of the plurality of projectors, a calculation for calculating unidirectional shape data indicating a shape-related value, which is a value related to a three-dimensional shape, for each pixel based on the pattern image acquired by capturing an image of the light irradiated from the projector to the substrate by the imaging unit;
the shape calculation unit executes a first integration calculation process to integrate the plurality of unidirectional shape data by calculating, for each pixel, an average of the shape-related values indicated by each of the plurality of unidirectional shape data, and calculate a three-dimensional shape of the measurement object;
In the first integrated calculation process, a three-dimensional measurement calculation device calculates the three-dimensional shape of the measurement object by a weighted average in which the shape-related values of the pixels included in the poorly illuminated area, among the shape-related values indicated by the unidirectional shape data, are multiplied by a weighting coefficient that is less than 1 and greater than or equal to 0. A three-dimensional measurement calculation device includes an object support section that supports a measurement object having a board and components mounted on the board, a pattern irradiation section that irradiates light of a predetermined pattern onto the measurement object supported by the object support section, and an imaging section that acquires a two-dimensional pattern image by imaging the light irradiated onto the measurement object from the pattern irradiation section, the three-dimensional measurement calculation device executing calculations for measuring a three-dimensional shape of the measurement object based on the pattern image acquired by a three-dimensional measurement device, the three-dimensional measurement device comprising:
a reference model acquisition unit that acquires a reference model indicating a component mounting area on the board on which the component is to be mounted and an outer shape of the component to be mounted in the component mounting area ;
a region calculation unit that calculates a poorly irradiated region, which is a secondary reflection region where light irradiated from the pattern irradiation unit and reflected by the component is incident on the board, based on irradiation direction information indicating a direction in which the light is irradiated onto the component by the three-dimensional measuring device and on the reference model;
the reference model indicates whether the part has a property of reflecting light;
The area calculation unit is a three-dimensional measurement calculation device that calculates the secondary reflection area for the part that is indicated by the reference model as reflecting light . 3. The three-dimensional measuring calculation device according to claim 1 or 2, wherein the reference model acquisition unit creates the reference model from at least one of inspection data indicating a standard for inspecting whether a positional relationship between the component mounted in the component mounting area and the component mounting area is appropriate and CAD (Computer-Aided Design) data indicating a configuration of the board. 4. The three-dimensional measurement calculation device according to claim 1, wherein the area calculation unit calculates the poorly irradiated area further based on a result of recognizing a position of the substrate that has been carried into the three-dimensional measurement device and supported by the object support unit. 5. The three-dimensional measuring calculation device according to claim 4 , wherein the position of the substrate supported by the object support section is recognized based on the result of imaging a fiducial mark affixed to the substrate by the imaging section. 6. The three-dimensional measurement calculation device according to claim 1 , wherein the area calculation section calculates the poorly irradiated area further based on a result of detecting the position of the component mounted in the component mounting area. 7. The three-dimensional measuring calculation device according to claim 6 , wherein the position of the component mounted in the component mounting area is detected based on a result of imaging the component. 2. The three-dimensional measurement calculation device according to claim 1 , wherein in the first integrated calculation process, the three-dimensional shape of the measurement object is calculated by a weighted average in which the shape-related values of the pixels included in the poorly-irradiated area are multiplied by a weighting coefficient of 0. 9. The three-dimensional measuring calculation device according to claim 1 , wherein the shape-related value is given by a product of a calculated value of the three-dimensional shape calculated based on the pattern image and a reliability of the calculated value. 10. The three-dimensional measuring calculation device according to claim 1, 8 or 9 , wherein the shape-related value is a calculated value of a three-dimensional shape calculated based on the pattern image. A three-dimensional measurement calculation device includes an object support section that supports a measurement object having a board and components mounted on the board, a pattern irradiation section that has a plurality of projectors that irradiate light of a predetermined pattern onto the measurement object supported by the object support section, and an imaging section that acquires a two-dimensional pattern image by imaging the light irradiated onto the measurement object from the pattern irradiation section, the three-dimensional measurement calculation device performing calculations for measuring a three-dimensional shape of the measurement object based on the pattern image acquired by a three-dimensional measurement device, the three-dimensional measurement device comprising:
a reference model acquisition unit that acquires a reference model indicating a component mounting area on the board on which the component is to be mounted and an outer shape of the component to be mounted in the component mounting area;
a region calculation unit that calculates at least one of poorly irradiated regions of a shadow region where a shadow of the component is cast on the board as a result of the light irradiated from the pattern irradiation unit to the component being blocked by the component and a secondary reflection region where light irradiated from the pattern irradiation unit and reflected by the component is incident on the board, based on irradiation direction information that indicates a direction in which light is irradiated to the component by the three-dimensional measuring device and on the reference model;
a shape calculation unit that calculates a plurality of unidirectional shape data by executing, for each of the plurality of projectors, a calculation for calculating unidirectional shape data indicating a shape-related value, which is a value related to a three-dimensional shape, for each pixel based on the pattern image acquired by capturing an image of the light irradiated from the projector to the substrate by the imaging unit;
Equipped with
the plurality of projectors irradiate light onto the component from different directions;
the irradiation direction information indicates, for each of the plurality of projectors, a direction in which light is irradiated from the projector to the component;
the region calculation unit calculates the poor illumination region for each of the plurality of projectors;
the shape calculation unit executes a first integration calculation process to integrate the plurality of unidirectional shape data by calculating, for each pixel, an average of the shape-related values indicated by each of the plurality of unidirectional shape data, and calculate a three-dimensional shape of the measurement object;
In the first integrated calculation process, the three-dimensional shape of the measurement object is calculated by a weighted average in which the shape-related values of the pixels included in the poorly-irradiated region among the shape-related values indicated by the one-directional shape data are multiplied by a weighting coefficient less than 1 and equal to or greater than 0;
the shape calculation unit executes both a second integrated arithmetic process, which integrates the plurality of unidirectional shape data by calculating for each pixel a representative value of the shape-related value indicated by the plurality of unidirectional shape data without weighting according to the poor irradiation region, and calculates a three-dimensional shape of the measurement object, and the first integrated arithmetic process, on the same measurement object, and executes an evaluation process, which evaluates a difference between the three-dimensional shapes of the measurement object calculated by each of the first integrated arithmetic process and the second integrated arithmetic process, on a predetermined number of measurement objects from which the pattern image was acquired by the three-dimensional measuring device, thereby executing a necessity determination to determine whether the first integrated arithmetic process is necessary;
After it is determined in the necessity judgment that the first integrated calculation process is unnecessary, the shape calculation unit calculates the three-dimensional shape of the measurement object by the second integrated calculation process without executing the first integrated calculation process. The calculation device for three-dimensional measurement according to claim 11 , further comprising a setting operation unit that accepts an operation for setting the predetermined number. A three-dimensional measurement program that causes a computer to function as the three-dimensional measurement calculation device according to any one of claims 1 to 12 . A recording medium on which the three-dimensional measurement program according to claim 13 is recorded so as to be readable by a computer. an object support unit that supports a measurement object having a substrate and a component mounted on the substrate;
a pattern irradiation unit that irradiates the measurement object supported by the object support unit with light of a predetermined pattern;
an imaging unit that captures an image of the light irradiated from the pattern irradiation unit onto the measurement object to obtain a two-dimensional pattern image;
13. A three-dimensional measuring apparatus comprising: a three-dimensional measuring calculation device according to claim 1, which executes calculations for measuring the three-dimensional shape of the measurement object based on the pattern image. A three-dimensional measurement calculation method for performing calculations for measuring a three-dimensional shape of a measurement object based on a pattern image acquired by a three-dimensional measurement device including: an object support unit that supports a measurement object having a board and components mounted on the board; a pattern irradiation unit that irradiates light of a predetermined pattern onto the measurement object supported by the object support unit; and an imaging unit that acquires a two-dimensional pattern image by imaging the light irradiated onto the measurement object from the pattern irradiation unit,
acquiring a reference model indicating a component mounting area on the board on which the component is to be mounted and an outer shape of the component to be mounted in the component mounting area ;
a region calculation unit calculating a poorly irradiated region, which is a secondary reflection region where light irradiated from the pattern irradiation unit and reflected by the component is incident on the board, based on irradiation direction information indicating a direction in which the light is irradiated onto the component by the three-dimensional measuring device and the reference model;
the pattern irradiation unit has a plurality of projectors that irradiate the component with light from different directions;
the irradiation direction information indicates, for each of the plurality of projectors, a direction in which light is irradiated from the projector to the component;
the region calculation unit calculates the poor illumination region for each of the plurality of projectors;
a shape calculation unit calculates a plurality of unidirectional shape data by executing, for each of the plurality of projectors, a calculation for calculating unidirectional shape data indicating a shape-related value, which is a value related to a three-dimensional shape, for each pixel based on the pattern image acquired by the imaging unit capturing the light irradiated from the projector to the substrate;
the shape calculation unit executes a first integration calculation process to integrate the plurality of unidirectional shape data by calculating, for each pixel, an average of the shape-related values indicated by each of the plurality of unidirectional shape data, and calculate a three-dimensional shape of the measurement object;
A calculation method for three-dimensional measurement, in which, in the first integrated calculation process, a three-dimensional shape of the measurement object is calculated by a weighted average in which the shape-related values of the pixels included in the poorly-irradiated area, among the shape-related values indicated by the one-directional shape data, are multiplied by a weighting coefficient that is less than 1 and is equal to or greater than 0 . A three-dimensional measurement calculation method for performing calculations for measuring a three-dimensional shape of a measurement object based on a pattern image acquired by a three-dimensional measurement device including: an object support unit that supports a measurement object having a board and components mounted on the board; a pattern irradiation unit that irradiates light of a predetermined pattern onto the measurement object supported by the object support unit; and an imaging unit that acquires a two-dimensional pattern image by imaging the light irradiated onto the measurement object from the pattern irradiation unit,
acquiring a reference model indicating a component mounting area on the board on which the component is to be mounted and an outer shape of the component to be mounted in the component mounting area ;
calculating a poorly irradiated area, which is a secondary reflection area where light irradiated from the pattern irradiation unit and reflected by the component is incident on the board, based on irradiation direction information indicating a direction in which light is irradiated onto the component by the three-dimensional measuring device and the reference model ;
the reference model indicates whether the part has a property of reflecting light;
A method of calculation for three-dimensional metrology , in which the secondary reflection area is calculated for the part that the reference model indicates when it reflects light .

Images