TWI843048B - Visual alignment system and visual alignment method thereof - Google Patents

Abstract

A visual alignment system and a visual alignment method are provided. The visual alignment system includes an image capture device and a shift device. The image capturing device has an image controller and an image capturing module. The shift device has a shift controller and a shift machine, and the shift controller controls the shift machine to move an object. The image controller configures and executes an automatic alignment according to a set parameter. The image controller controls the image capturing module to obtain the object image. The image controller calculates a correction coordinate according to the object image and returns the correction coordinate to the shift device. The shift controller controls the shift machine to move the object to the standard position according to the corrected coordinate.

Description

Visual counterpoint system and visual counterpoint method thereof

The present invention relates to a positioning system and method, and in particular to a visual positioning system and a visual positioning method thereof.

In the current automated production process, objects need to be aligned. However, the commonly used mechanical alignment method is not applicable to different types of objects, such as wafers, glass substrates or PCB carriers.

The technical problem to be solved by the present invention is to provide a visual alignment system and method to address the deficiencies of the existing technology.

The present invention provides a visual alignment system, including an image capture device and a mobile device. The image capture device has an image controller and an image capture module. The image controller is electrically connected to the image capture module. The mobile device has a mobile controller and a mobile machine, the mobile controller is electrically connected to the image controller and the mobile machine, and the mobile controller controls the mobile machine to move an object to an alignment area.

The image controller is configured to perform an automatic alignment according to a setting parameter: the image controller controls the image capture module to take a picture of an object image of the object located in the alignment area; the image controller obtains a first image feature position of a first image feature and a second image feature position of a second image feature in the object image according to the object image; the image controller calculates a correction offset from a first standard image feature position and a second standard image feature position according to the first image feature position and the second image feature position; the image controller calculates a correction coordinate of the object according to the correction offset; the image controller returns the correction coordinate to the mobile controller of the mobile device, so that the mobile controller controls the mobile machine to move the position of the object according to the correction coordinate, and the first image feature position in the position after the object moves is the same as the first standard image feature position and the second image feature position is the same as the second standard image feature position.

The present invention provides a visual alignment method applicable to an image capture device and a mobile device, comprising: when the mobile device moves an object to an alignment area, the image capture device performs an automatic alignment according to a setting parameter configuration; the image capture device takes a picture of an object image located in the alignment area; the image capture device obtains a first image feature position of a first image feature and a second image feature position of a second image feature in the object according to the object image; the image capture device performs an automatic alignment according to a setting parameter configuration; The image feature position and the second image feature position respectively calculate a correction offset with a first standard image feature position and a second standard image feature position; the image capture device calculates a correction coordinate of the object according to the correction offset; and the image capture device returns the correction coordinate to the mobile device, so that the mobile controller moves the position of the object according to the correction coordinate, and the first image feature position of the object after the movement is the same as the first standard image feature position and the second image feature position is the same as the second standard image feature position.

In summary, the visual alignment system and visual alignment method provided by the embodiments of the present invention can be widely used in various alignment occasions and can be quickly and accurately aligned without the need for complex alignment mechanism design. The hardware adjustment time can be reduced through parameter setting.

To further understand the features and technical contents of the present invention, please refer to the following detailed description and drawings of the present invention. However, the drawings provided are only for reference and description and are not used to limit the present invention.

1: Visual alignment system

10: Mobile devices

101:Move controller

103: Mobile machine

20: Image capture equipment

201: Image controller

203: Image capture module

2031: First Camera

2033: Second Camera

205: Light source controller

207: Light source

D1: First direction

D2: Second direction

D3: Third direction

3: Objects

31: First image feature

33: Second image feature

5: Positioning area

S801: Load system setting parameters

S803: Load visual setting parameters

S805: Receive mobile device communication command

S807: Recipe switching

S809: Execute recipe switching operation

S811: Auto focus

S813: Perform auto focus operation

S815: Automatic calibration

S817: Perform automatic calibration operation

S819: Standard position record

S821: Perform standard position recording operation

S823: Automatic alignment

S825: Perform automatic alignment operation

S901: Receive recipe data from mobile device

S903: Determine whether there is corresponding data

S905: Change recipe data

S907: Return error message

S909: Completed

S1001: Receive mobile device location information

S1003: Camera capture

S1005: Determine whether the camera captures the image successfully

S1007: Identify image focus status

S1009: Update mobile device location information

S1011: Wait for the mobile device to move to completion

S1013: Determine whether focus is completed

S1015: Return error message

S1017: Completed

S1101: Receive mobile device location information

S1103: Camera capture

S1105: Determine whether the camera captures the image successfully

S1107: Record image feature locations

S1109: Update mobile device location information

S1111: Wait for the mobile device to move to completion

S1113: Determine whether the calibration is completed

S1115: Save calibration information

S1117: Return error message

S1119: Completed

S1201: Record the standard position of the capture

S1203: Camera capture

S1205: Determine whether the camera captures the image successfully

S1207: Record image feature locations

S1209: Return error message

S1211: Completed

S1301: Receive mobile device alignment command

S1303: Whether to automatically adjust the light source

S1305: Dimming completed

S1307: Determine single camera

S1309: Single camera imaging

S1311: Dual-camera imaging

S1313: Determine whether the camera captures the image successfully

S1315: Determine whether the camera is fixed

S1317: Camera fixed calculation

S1319: Determine whether it is a panoramic view

S1321: Get the feature position in the panoramic image

S1323: Calculate the offset correction

S1325: Get the dual image feature position

S1327: Calculate the offset

S1329: Calculation of objects requires correction of coordinates

S1331: Camera mobile computing

S1333: Get the dual image feature position

S1335: Calculate the offset correction

S1337: Calculate the correct coordinates required for grabbing

S1339: Return calculation results

S1341: Completed

S1343: Return error message

Figure 1 is a block diagram of a visual alignment system provided in an embodiment of the present invention.

Figure 2 is a system architecture diagram of the first dual vision alignment mode provided by an embodiment of the present invention.

Figure 3 is a system architecture diagram of the second binaural alignment mode provided by the embodiment of the present invention.

Figure 4 is a system architecture diagram of the third binocular vision alignment mode provided by the embodiment of the present invention.

Figure 5 is a system architecture diagram of the first single-vision alignment mode provided in an embodiment of the present invention.

Figure 6 is a system architecture diagram of the second single-vision alignment mode provided in an embodiment of the present invention.

FIG7 is a system architecture diagram of the third single-vision alignment mode provided in an embodiment of the present invention.

Figure 8 is a flow chart of a visual alignment method provided in an embodiment of the present invention.

Figure 9 is a flow chart of the recipe switching operation provided in the embodiment of the present invention.

Figure 10 is a flowchart of the automatic focus operation provided in the embodiment of the present invention.

Figure 11 is a flow chart of the automatic correction operation provided in the embodiment of the present invention.

Figure 12 is a flow chart of a standard location recording operation provided in an embodiment of the present invention.

Figure 13 is a flow chart of the automatic alignment operation provided in the embodiment of the present invention.

The following is a specific embodiment to illustrate the implementation of the present invention. The technical personnel in this field can understand the advantages and effects of the present invention from the content provided in this manual. The present invention can be implemented or applied through other different specific embodiments. The details in this manual can also be modified and changed based on different viewpoints and applications without deviating from the concept of the present invention. In addition, the attached drawings of the present invention are only for simple schematic illustration and are not depicted according to actual size. Please note in advance. The following implementation will further explain the relevant technical content of the present invention in detail, but the content provided is not intended to limit the scope of protection of the present invention.

It should be understood that although the terms "first", "second", "third" and so on may be used in this article to describe various components or signals, these components or signals should not be limited by these terms. These terms are mainly used to distinguish one component from another component, or one signal from another signal. In addition, the term "or" used in this article may include any one or more combinations of the related listed items depending on the actual situation.

The present invention provides a visual alignment system and a visual alignment method. The visual alignment described herein is to determine whether the placement position of an object meets the standard position through visual images. If not, the offset between the current placement position of the object and the standard position can be further calculated, and then the placement position of the object is automatically moved and aligned with the standard position according to the offset, so that multiple objects can be placed in order in this way. The objects described herein are, for example, wafers, glass substrates or PCB carriers, but are not limited to them.

[Hardware architecture of visual alignment system]

Please refer to FIG. 1 , which is a block diagram of a visual alignment system provided in an embodiment of the present invention. The visual alignment system 1 described in the present embodiment, for example, includes a mobile device 10 and an image capture device 20, wherein the mobile device 10 and the image capture device 20 can transmit data to each other through a connection, for example, in a wired or wireless manner. The mobile device 10, for example, moves an object, and the moving direction described here is, for example, the X-axis, Y-axis, and Z-axis of the three-dimensional coordinate system. Furthermore, in addition to controlling the movement of the object on the X-axis, Y-axis, and Z-axis, the mobile device 10 can also rotate the object. The image capture device 20, for example, captures an image of an object, thereby obtaining an object image of the object, and then performs image analysis and comparison based on the object image to obtain a correction offset, and then the correction coordinates required for the object can be relatively calculated based on the correction offset, and the correction coordinates can be provided by the image capture device 20 to the mobile device 10. The mobile device 10 can finally move the object according to the correction coordinates and enable the object to be moved to the standard position.

In one embodiment, the mobile device 10 includes, for example, a mobile controller 101 and a mobile station 103, wherein the mobile controller 101 is electrically connected to the mobile station 103. The image capture device 20 includes, for example, an image controller 201, an image capture module 203, a light source controller 205, and a light source 207, wherein the image controller 201 is electrically connected to the image capture module 203 and the light source controller 205, and the light source controller 205 is electrically connected to the light source 207.

The mobile controller 101 and the image controller 201 can transmit data to each other, and the mobile controller 101 can control the mobile platform 103 to control the moving direction of the object. The mobile platform 103 is, for example, a robot arm, and the robot arm can move the object by grabbing, sucking or clamping. The image controller 201 can control the image capture module 203 to take a picture of the object. The image capture module 203 is, for example, one or more cameras. The light source controller 205 controls the brightness of the light source 207 according to the control of the image controller 201.

Furthermore, the image controller 201 can perform an automatic alignment according to a setting parameter configuration. For example, the image controller 201 controls the image capture module 203 to take a photo of an object to obtain an object image, and then the image controller 201 analyzes the object image to obtain a first image feature position of a first image feature and a second image feature position of a second image feature in the object image. Afterwards, the image controller 201 calculates the position difference between the first image feature position and the second image feature position and a first standard image feature position and a second standard image feature position respectively to obtain a correction offset. The image controller 201 then calculates a correction coordinate that the object needs to be moved according to the correction offset. Then the image controller 201 returns the correction coordinates to the motion controller 101, and the motion controller 101 can move the position of the object according to the correction coordinates, and the first image feature position of the object after being moved is the same as the first standard image feature position and the second image feature position is the same as the second standard image feature position, thereby achieving the alignment control of the object.

It is worth noting that the above-mentioned setting parameters can be input into the image capture device 20 according to actual needs of personnel, and then the visual alignment system 1 can be applied to different alignment methods in various industries. In particular, the setting parameters include, for example, a system setting parameter and a visual setting parameter, wherein the system setting parameter is a setting mode related to the image capture module 203, and the visual setting parameter is a working parameter related to the image capture module 203 obtaining the object image.

In one embodiment, the setting mode of the image capture module 203 is, for example, a first binocular alignment mode, a second binocular alignment mode, a third binocular alignment mode, a fourth monocular alignment mode, a fifth monocular alignment mode, or a sixth monocular alignment mode. The following will give examples to illustrate each setting mode.

Please refer to FIG. 2, which is a system architecture diagram of the first dual-vision alignment mode provided by the embodiment of the present invention. Here, the first dual-vision alignment mode means that the image capture device 20 is fixedly installed on the mobile device 10, and the image capture module 203 includes a first camera 2031 and a second camera 2033. That is to say, when the mobile device 10 moves in the first direction D1, the second direction D2, or rotates an angle in the rotation direction D3, the image capture module 203 moves or rotates synchronously.

In addition, after the object 3 is moved to the alignment area 5 by the mobile device 10, the object 3 is relatively placed in the alignment area 5 below the image capture module 203, so that the first camera 2031 and the second camera 2033 can smoothly take pictures of the object 3. In addition, it is worth noting that the object 3 itself includes a first image feature 31 and a second image feature 33, and the first image feature 31 and the second image feature 33 are, for example, arranged on both sides of the object 3, and the arrangement positions of the first camera 2031 and the second camera 2033 roughly correspond to the first image feature 31 and the second image feature 33 in the object 3, so that the first camera 2031 can obtain the first image feature 31 when taking pictures of the object 3, and the second camera 2033 can obtain the second image feature 33 when taking pictures of the object 3.

Please refer to FIG. 3, which is a system architecture diagram of the second dual-vision alignment mode provided by the embodiment of the present invention. Here, the second dual-vision alignment mode means that the image capture device 20 is separately arranged outside the mobile device 10, and the image capture module 203 includes a first camera 2031 and a second camera 2033. That is to say, when the mobile device 10 moves in the first direction D1, the second direction D2, or rotates an angle in the rotation direction D3, the image capture module 203 is fixed and does not move.

In addition, the object 3 is also placed in the alignment area 5 below the image capture module 203, so that the first camera 2031 and the second camera 2033 can smoothly take pictures of the object 3. The object 3 itself also includes a first image feature 31 and a second image feature 33, which are, for example, arranged on both sides of the object 3, and the first camera 2031 and the second camera 2033 are arranged at positions roughly corresponding to the first image feature 31 and the second image feature 33 in the object 3, so that the first camera 2031 can obtain the first image feature 31 when taking pictures of the object 3, and the second camera 2033 can obtain the second image feature 33 when taking pictures of the object 3.

Please refer to FIG. 4, which is a system architecture diagram of the third dual-vision alignment mode provided by the embodiment of the present invention. The third dual-vision alignment mode described herein refers to the image capture device 20 being separately disposed outside the mobile device 10, and the image capture module 203 includes a first camera 2031 and a second camera 2033. The mobile device 10 captures the object 3 and cooperates with the moving object 3 so that the image capture module 203 can simultaneously capture the object 3 and then align it.

Specifically, the first camera 2031 and the second camera 2033 described in FIG. 4 are fixedly installed below the mobile device 10, and the object 3 is located at a position where the image capture module 203 can take pictures. For example, after the mobile device 10 grabs the object 3, it can move in the first direction D1, the second direction D2, or rotate an angle in the rotation direction D3. At this time, the image capture module 203 is fixed. In addition, the installation positions of the first camera 2031 and the second camera 2033 roughly correspond to the first image feature 31 and the second image feature 33 in the object 3, so that the first camera 2031 can obtain the first image feature 31 when taking pictures of the object 3, and the second camera 2033 can obtain the second image feature 33 when taking pictures of the object 3.

Please refer to FIG. 5, which is a system architecture diagram of the first single vision alignment mode provided by the embodiment of the present invention. Here, the first single vision alignment mode means that the image capture device 20 is fixedly installed on the mobile device 10, and the image capture module 203 includes a first camera 2031. That is to say, when the mobile device 10 moves in the first direction D1, the second direction D2, or rotates an angle in the rotation direction D3, the image capture module 203 moves or rotates synchronously.

In addition, the object 3 is relatively placed in the alignment area 5 below the image capture module 203, so that the first camera 2031 can smoothly take pictures of the object 3. In addition, it is worth noting that the object 3 itself includes a first image feature 31 and a second image feature 33, which are, for example, arranged on both sides of the object 3, and the first camera 2031 is controlled by the mobile device 10 to move, so that the first camera 2031 can obtain the first image feature 31 and the second image feature 33 respectively when taking pictures of the object 3.

Please refer to FIG. 6, which is a system architecture diagram of the second single-vision alignment mode provided in the embodiment of the present invention. Here, the second single-vision alignment mode means that the image capture device 20 is separately disposed outside the mobile device 10, and the image capture module 203 includes a first camera 2031. That is to say, when the mobile device 10 moves, the image capture module 203 is fixed.

In addition, the object 3 is captured by the mobile device 10 and moves with the mobile device 10. Specifically, the object 3 is located at a position where the image capture module 203 can take a photo. For example, after the mobile device 10 captures the object 3, it can move in the first direction D1, the second direction D2, or rotate in the rotation direction D3 by an angle. At this time, when the first camera 2031 takes a photo of the object 3, the first image feature 31 and the second image feature 33 can be obtained respectively.

Please refer to FIG. 7, which is a system architecture diagram of the third single-vision alignment mode provided by the embodiment of the present invention. The third single-vision alignment mode described herein refers to that the image capture device 20 is separately disposed outside the mobile device 10, and the image capture module 203 includes a first camera 2031. After the mobile device 10 grabs the object, the object 3 is placed in the alignment area 5, and the alignment area 5 is, for example, disposed below the shooting angle of the first camera 2031. Specifically, the first camera 2031 described in FIG. 7 is a panoramic photograph of the object, that is, the first camera 2031 can obtain the first image feature 31 and the second image feature 33 in the object 3 in one shot.

[Visual alignment control implementation example]

Please refer to Figure 8. Figure 8 is a flow chart of a visual alignment method provided in an embodiment of the present invention. The flow chart shown in Figure 8 is illustrated by the structure of Figure 1, but is not limited thereto. The process shown in Figure 8 includes the following steps, for example.

In step S801, the image capture device 20 loads system setting parameters.

In step S803, the image capture device 20 loads the visual setting parameters.

In step S805, a communication command from the mobile device 10 is received.

In step S807, it is determined whether it is a recipe switch. If it is determined to be yes, step S809 is executed, and if it is determined to be no, step S811 is executed. The recipe switch described herein is, for example, that the image capture device 20 selects a reference data corresponding to the current object 3 according to the recipe switch instruction provided by the mobile device 10. The image capture device 20 can then control the mobile device 10 according to the reference data and the related setting parameters of S801 and S803 to perform related operations such as auto-focus, auto-calibration, standard position recording or auto-alignment on the object 3.

In step S809, the image capture device 20 performs a recipe switching operation. The recipe switching operation will be described later.

In step S811, it is determined whether it is auto focus. If it is determined to be yes, step S813 is executed, and if it is determined to be no, step S815 is executed. The auto focus mentioned here refers to the image capture module 203 in the image capture device 20 performing an auto focus operation on the object 3, so that it can be used for subsequent automatic alignment of the object 3.

In step S813, the image capture device 20 performs an auto focus operation. The auto focus operation will be described later.

In step S815, it is determined whether it is automatic calibration. If it is determined to be yes, step S817 is executed, and if it is determined to be no, step S819 is executed. The automatic calibration mentioned here refers to the related operations of performing camera calibration on the image capture module 203.

In step S817, the image capture device 20 performs an automatic calibration operation. The automatic calibration operation will be described later.

In step S819, determine whether it is a standard position record. If it is determined to be yes, execute step S821, and if it is determined to be no, execute step S823. The standard position record mentioned here means that object 3 needs to be placed in the correct position for subsequent alignment and comparison.

In step S821, the image capture device 20 performs a standard position recording operation. The standard position recording will be described later.

In step S823, it is determined whether it is automatic alignment. If it is determined to be yes, step S825 is executed, and if it is determined to be no, step S805 is executed. The automatic alignment mentioned here refers to calculating the correction coordinates required for the current object 3 to be adjusted to the standard position.

In step S825, the image capture device 20 performs an automatic alignment operation. The automatic alignment operation will be described later.

It is worth noting that when the vision alignment system 1 performs alignment-related operations, it can first determine the alignment architecture to be used by setting parameters (such as system setting parameters and vision setting parameters). The system setting parameters are, for example, a setting mode related to the image capture module 203 (such as the architecture shown in Figures 2 to 7), and the vision setting parameters are, for example, a working parameter related to the image capture module 203 obtaining the object image. The working parameter is, for example, the position, size or shape of the first image feature 31 and the second image feature 33 set on the object 3, or the working parameter can also be the alignment height between the image capture module 203 and the object 3. When the setting parameters are set, the mobile device 10 and the image capture device 20 can communicate with each other. For example, the image capture device 20 can determine whether it has received a recipe switching command, an auto focus command, an auto correction command, a standard position recording command or an auto alignment command issued by the mobile device 10, thereby achieving that the visual alignment system 1 can be used in various alignment occasions and effectively simplify the complexity of the alignment operation for personnel.

Please refer to Figure 9. Figure 9 is a flow chart of a recipe switching operation provided in an embodiment of the present invention. The flow chart shown in Figure 9 is illustrated by the structure of Figure 1, but is not limited thereto. The process shown in Figure 9 is executed on the image capture device 20, for example, and includes the following steps.

In step S901, the image capture device 20 receives the recipe data of the mobile device 10. The recipe data herein refers to, for example, the size of the object 3.

In step S903, the image capture device 20 determines whether there is corresponding data. If the determination is yes, step S905 is executed, and if the determination is no, step S907 is executed.

In step S905, the image capture device 20 changes the recipe data.

In step S907, the image capture device 20 returns an error message.

In step S909, the image capture device 20 completes the recipe switching operation.

Please refer to Figure 10. Figure 10 is a flowchart of an autofocus operation provided in an embodiment of the present invention. The flowchart shown in Figure 10 is illustrated by the structure of Figure 1, but is not limited thereto. The process shown in Figure 10 is executed on the image capture device 20, for example, including the following steps.

In step S1001, the location information of the mobile device 10 is received. When the image capture module 203 is set on the mobile module 10, the distance between the image capture module 203 and the object 3 will change due to the movement of the mobile device 10.

In step S1003, the camera captures an image. When the image capture module 203 is operated in the dual vision alignment mode, the camera includes the first camera 2031 and the second camera 2033. When the image capture module 203 is operated in the single vision alignment mode, the camera includes the first camera 2031.

In step S1005, it is determined whether the camera captures the image successfully. If it is determined to be yes, step S1007 is executed, and if it is determined to be no, step S1015 is executed. The successful capture mentioned here means that the object is photographed successfully.

In step S1007, the image focus state is identified.

In step S1009, the location information of the mobile device 10 is updated.

In step S1011, wait for the mobile device 10 to complete moving.

In step S1013, determine whether the focus is completed. If it is determined to be yes, execute step S1017, and if it is determined to be no, execute step S1003.

In step S1015, an error message is returned.

In step S1017, complete.

Please refer to Figure 11. Figure 11 is a flowchart of the automatic correction operation provided by the embodiment of the present invention. The flowchart shown in Figure 11 is illustrated by the structure of Figure 1, but is not limited thereto. The process shown in Figure 11 is executed on the image capture device 20, for example, including the following steps.

In step S1101, the location information of the mobile device 10 is received.

In step S1103, the camera captures an image.

In step S1105, determine whether the camera captures the image successfully. If it is determined to be yes, execute step S1107, and if it is determined to be no, execute step S1117.

In step S1107, the image feature position is recorded. Here, the image feature is, for example, a first image feature position of the first image feature 31 or a second image feature position of the second image feature 33.

In step S1109, the location information of the mobile device 10 is updated.

In step S1111, wait for the mobile device 10 to complete moving.

In step S1113, it is determined whether the calibration is completed. If it is determined to be yes, step S1115 is executed, and if it is determined to be no, step S1101 is executed. The calibration completion mentioned here refers to, for example, camera internal parameters, camera external parameters and distortion parameters. The camera internal parameters are, for example, the projection of the camera coordinate system to the image coordinate system, and the camera external parameters are, for example, the coordinate conversion of the world coordinate system to the camera coordinate system.

In step S1115, the calibration information is stored. Based on this calibration information, the image capture module 203 can perform subsequent normal photography.

In step S1117, an error message is returned.

In step S1119, complete.

Please refer to FIG. 12. FIG. 12 is a flowchart of a standard position recording operation provided in an embodiment of the present invention. The flowchart shown in FIG. 12 is illustrated by taking the structure of FIG. 1 as an example, but is not limited thereto. The process shown in FIG. 12 is executed on the image capture device 20, for example, including the following steps.

In step S1201, the capture standard position is recorded. Here, the image capture module 203 is activated for subsequent image capture.

In step S1203, the camera captures an image.

In step S1205, determine whether the camera has successfully captured the image. If it is determined to be yes, execute step S1207, and if it is determined to be no, execute step S1209.

In step S1207, the image feature position is recorded. Here, for example, the camera in the image capture module 203 is used to take a picture of the object 3 to obtain the image feature position in the object 3, and the image feature position is used as the standard position for subsequent alignment of other objects. Specifically, the image feature position is, for example, the first standard image feature position of the first image feature 31 in the object 3 and the second standard image feature position of the second image feature 33.

In step S1209, an error message is returned.

In step S1211, complete.

Please refer to Figure 13. Figure 13 is a flowchart of the automatic alignment operation provided by the embodiment of the present invention. The flowchart shown in Figure 13 is illustrated by the structure of Figure 1, but is not limited thereto. The process shown in Figure 13 is executed on the image capture device 20, for example, including the following steps.

In step S1301, the alignment command of the mobile device 10 is received. When the image capture device 20 receives the alignment command provided by the mobile device 10, the subsequent alignment operation is automatically started.

In step S1303, it is determined whether the light source 207 is automatically adjusted. If it is determined to be yes, step S1305 is executed, and if it is determined to be no, step S1307 is executed. When the image capture device 20 turns on the automatic adjustment of the light source 207, the image controller 201 can adjust the brightness of the light source 207 by controlling the light source controller 205.

In step S1305, dimming is completed. Here, dimming completion means that the light source controller 205 adjusts the brightness of the light source 207 to meet a preset brightness or the brightness required by the on-site environment.

In step S1307, determine whether it is a single-camera camera. If it is determined to be yes, execute step S1309, and if it is determined to be no, execute step S1311. In this case, the image capture device 20 can know whether the image capture device 20 uses a single camera or a dual camera according to the system setting parameters.

In step S1309, a single camera captures an image. Here, the image controller 201 controls the first camera 2031 to take a photo and capture an image.

In step S1311, dual cameras capture images. Here, the image controller 201 controls the first camera 2031 and the second camera 2033 to take photos and capture images.

In step S1313, determine whether the camera successfully captures the image. If it is determined to be yes, execute step S1315, and if it is determined to be no, execute step S1343.

In step S1315, it is determined whether the camera is fixed. If it is determined to be yes, step S1317 is executed, and if it is determined to be no, step S1331 is executed. In this case, the image capture device 20 can know whether the camera mounting method in the image capture device 20 is fixed according to the system setting parameters.

In step S1317, the camera is fixed for calculation.

In step S1319, it is determined whether it is a panoramic view. If it is determined to be yes, step S1321 is executed, and if it is determined to be no, step S1325 is executed. In this case, the image controller 201 can know whether the system architecture of the visual alignment system 1 belongs to panoramic photography according to the system setting parameters, such as the architecture shown in Figure 7 above.

In step S1321, the feature position in the panoramic image is obtained. Here, the image controller 201 obtains the feature position in the object 3 through the first camera 2031 for panoramic photography. The feature position is the first image feature position of the first image feature 31 and the second image feature position of the second image feature 33 in the object image.

In step S1323, the correction offset is calculated. Here, the image controller 201 calculates the distance difference between the first image feature position and the first standard image feature position and the distance difference between the second image feature position and the second standard image feature position to obtain the correction offset.

In step S1325, the dual image feature positions are obtained. The image controller 201 obtains the first image feature position of the first image feature 31 in the object 3 through the first camera 2031 and the second image feature position of the second image feature 33 through the second camera 2033.

In step S1327, the offset correction is calculated. This calculation method is synchronized with step S1323.

In S1329, the coordinates required to be corrected for object 3 are calculated. The image controller 201 calculates the corrected coordinates of object 3 according to the correction offset. The corrected coordinates are mainly used to control the position of the moving object so that the first image feature position of object 3 after moving is the same as the first standard image feature position and the second image feature position is the same as the second standard image feature position.

In step S1331, the camera movement is calculated. The image controller 201 relatively calculates the change in the shooting position of the first camera 2031 and the second camera 2033 according to the movement of the mobile device 10.

In step S1333, the dual image feature position is obtained. When dual cameras are used for image capture, the image controller 201 obtains the first image feature position of the first image feature 31 in the object 3 through the first camera 2031 and the second image feature position of the second image feature 33 through the second camera 2033. Alternatively, when single camera image capture is used, the image controller 201 obtains the first image feature position of the first image feature 31 in the object 3 through the first camera 2031, and then moves the first camera 2031 to the second image feature through the mobile device 10 to obtain the second image feature position of the second image feature 33.

In step S1335, the offset correction is calculated. This calculation method is synchronized with step S1323.

In step S1337, the correction coordinates required for grabbing are calculated. This calculation method is synchronized with step S1329.

In step S1339, the calculation result is returned. After the image controller 201 calculates the correction coordinates, the image controller 201 can return the correction coordinates to the mobile controller 101, and the mobile controller 101 controls the mobile machine 103 to move the object 3 to the standard position according to the correction coordinates.

In step S1341, complete.

In one embodiment, the image controller 201 and the mobile controller 101 may be, for example, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or a system on a chip (SOC), or any combination thereof, and may be combined with other related circuit components and firmware to implement the above functional processes.

[Beneficial effects of the embodiment]

The visual alignment system and visual alignment method provided by the present invention can find the correction coordinates of the object that needs to be aligned to the standard position through image comparison, which can make the alignment application places wider and more diverse. Furthermore, by setting parameters, it can effectively simplify the inconvenience of hardware adjustment for personnel, thereby effectively improving the execution efficiency of the alignment work.

The above contents are only the preferred feasible embodiments of the present invention, and do not limit the scope of the patent application of the present invention. Therefore, all equivalent technical changes made by using the contents of the description and drawings of the present invention are included in the scope of the patent application of the present invention.

1: Visual alignment system

10: Mobile devices

101:Move controller

103: Mobile machine

20: Image capture equipment

201: Image controller

203: Image capture module

205: Light source controller

207: Light source

Claims (10)

A visual alignment system includes: an image capture device having an image controller and an image capture module, the image controller being electrically connected to the image capture module; and a mobile device having a mobile controller and a mobile platform, the mobile controller being electrically connected to the image controller and the mobile platform, the mobile controller controlling the mobile platform to move an object to an alignment area; wherein the image controller is configured to perform an automatic alignment according to a setting parameter: the image controller controls the image capture module to take a picture of an object image of the object located in the alignment area; the image controller obtains the object image according to the object image a first image feature position of a first image feature and a second image feature position of a second image feature in the image; the image controller calculates a correction offset from a first standard image feature position and a second standard image feature position according to the first image feature position and the second image feature position; the image controller calculates a correction coordinate of the object according to the correction offset; the image controller returns the correction coordinate to the mobile controller of the mobile device, so that the mobile controller controls the mobile platform to move the position of the object according to the correction coordinate, and the first image feature position in the position after the object moves The first standard image feature position and the second image feature position are the same as the second standard image feature position; wherein the setting parameter includes a system setting parameter and a visual setting parameter, wherein the system setting parameter is a setting mode related to the image capture module, and the visual setting parameter is a working parameter related to the image capture module obtaining the image of the object, wherein the setting mode is the setting method of the image capture device and the mobile device and the number of camera settings of the image capture device; wherein the image capture device selects a corresponding image according to a recipe switching instruction provided by the mobile device. A reference data for alignment, the image capture device controls the mobile device according to the reference data for alignment and the setting parameter to automatically calibrate the object; wherein the image capture device performs the automatic calibration, including: the image capture device receives a mobile device position information provided by the mobile device; the image capture device takes a picture of the object to obtain the object image; the image capture device records the position information of the first image feature and the second image feature in the object image; the image capture device updates the mobile device position information according to the position information of the first image feature and the second image feature, until the calibration is completed. A visual alignment system as described in claim 1, wherein the working parameter is the position, size or shape of the first image feature and the second image feature set on the object, or the working parameter is the alignment height between the image capture module and the object. The visual alignment system as described in claim 2, wherein the installation mode is a first dual-vision alignment mode, a second dual-vision alignment mode, a third dual-vision alignment mode, a fourth single-vision alignment mode, a fifth single-vision alignment mode or a sixth single-vision alignment mode, including: wherein the first dual-vision alignment mode is that the image capture device is fixedly arranged on the mobile device, and the image capture module includes a first camera and a second camera; wherein the second dual-vision alignment mode is that the image capture device is separately arranged outside the mobile device, and the image capture module includes a first camera and a second camera; wherein the third dual-vision alignment mode is that the image capture device is separately arranged on the mobile device, and the image capture module includes a first camera and a second camera. The image capturing module includes a first camera and a second camera, and the mobile device captures the object so that the image capturing device can simultaneously capture the object and then align it; wherein the fourth single vision alignment mode is that the image capturing device is fixedly arranged on the mobile device, and the image capturing module only includes a first camera; wherein the fifth single vision alignment mode is that the image capturing device is separately arranged outside the mobile device, and the image capturing module only includes a first camera, and the mobile device captures the object so that the first camera can capture the image and then align it; wherein the sixth single vision alignment mode is that the image capturing device is separately arranged outside the mobile device, and the image capturing module only includes a first camera, and the first camera captures the object in a panoramic manner and then aligns it. The visual alignment system as described in claim 2, wherein the image capture device further includes controlling the mobile device according to the alignment reference data and the setting parameters to perform automatic focusing, standard position recording or automatic alignment operations on the object. The visual alignment system as described in claim 4, wherein the recipe switching instruction is related to the size of the object, and when the image capture device performs the auto focus or the auto calibration, the image capture device relatively adjusts the distance between the mobile device and the object according to the mobile device position information provided by the mobile device and the result of the image capture device taking a picture of the object, and the image capture device When the standard position recording is completed, the image capture device obtains the first standard image feature position and the second standard image feature position. When executing the automatic alignment, the image capture device relatively performs a single-camera photo or a dual-camera photo according to the setting parameters, and relatively performs a fixed alignment calculation or a mobile alignment calculation according to whether the image capture module is fixed. A visual alignment method is applicable to an image capture device and a mobile device, comprising: the image capture device receives an input of a setting parameter, the setting parameter includes a system setting parameter and a visual setting parameter, wherein the system setting parameter is a setting mode associated with an image capture module in the image capture device, and the visual setting parameter is a working parameter associated with the image capture module obtaining an image of an object, wherein the setting mode is a setting method of the image capture device and the mobile device and the number of camera settings of the image capture device; in the mobile device When the object is moved to an alignment area, the image capture device performs an automatic alignment according to the setting parameter configuration; the image capture device takes a picture of an object image of the object located in the alignment area; the image capture device obtains a first image feature position of a first image feature and a second image feature position of a second image feature in the object according to the object image; the image capture device calculates a correction offset from a first standard image feature position and a second standard image feature position according to the first image feature position and the second image feature position; the image capture device The image capture device calculates a correction coordinate of the object according to the correction offset; and the image capture device returns the correction coordinate to the mobile device, so that a mobile controller of the mobile device moves the position of the object according to the correction coordinate, and the first image feature position of the object after the movement is the same as the first standard image feature position and the second image feature position is the same as the second standard image feature position; wherein the image capture device selects a reference data corresponding to the object according to a recipe switching instruction provided by the mobile device, and the image capture device selects a reference data corresponding to the object according to the recipe switching instruction provided by the mobile device. The reference data and the setting parameters are used to control the mobile device to automatically calibrate the object; wherein the image capture device performs the automatic calibration, including: the image capture device receives a mobile device position information provided by the mobile device; the image capture device takes a photo of the object to obtain the object image; the image capture device records the position information of the first image feature and the second image feature in the object image; the image capture device updates the mobile device position information according to the position information of the first image feature and the second image feature until the calibration is completed. The visual alignment method as described in claim 6 further includes: the working parameter is the position, size or shape of the first image feature and the second image feature set on the object, or the working parameter is the alignment height between the image capture module and the object; the image capture device controls the mobile device according to the alignment reference data and the setting parameter to perform automatic focus, standard position recording or the automatic alignment operation on the object. The visual alignment method as described in claim 7, wherein the image capture device performs the auto focus, including: the image capture device receives a mobile device location information provided by the mobile device; the image capture device takes a photo of the object to obtain the object image; the image capture device identifies the focus state of the object image; the image capture device updates the mobile device location information according to the focus state until the focus is completed. The visual alignment method as described in claim 7, wherein the image capture device performs standard position recording, including: recording a positioning standard position of the mobile device; the image capture device takes a photo of the object to obtain the object image; the image capture device records the position information of the first image feature and the second image feature in the object image. The visual alignment method as described in claim 7, wherein the image capture device performs single-camera photography or dual-camera photography relatively according to the setting parameters when performing the automatic alignment, and performs a fixed alignment calculation or a mobile alignment calculation relatively according to whether the image capture module is fixed; wherein the fixed alignment calculation includes: when using a single-camera photography, the image capture device obtains the object image of the object in a panoramic photography manner; when using dual-camera photography, a first camera in the image capture device photographs the first image feature position of the first image feature in the object, and a second camera in the image capture device photographs the second image feature position of the second image feature in the object; wherein the The mobile alignment calculation includes: when using a single camera to take pictures, the mobile device moves the image capture device, and a first camera in the image capture device takes a picture of the first image feature position of the first image feature in the object, and the mobile device then moves the image capture device, and the first camera in the image capture device takes a picture of the second image feature position of the second image feature in the object; when using a dual camera to take pictures, the mobile device moves the image capture device, and a first camera in the image capture device takes a picture of the first image feature position of the first image feature in the object, and the second camera in the image capture device takes a picture of the second image feature position of the second image feature in the object.

Images