WO2025013294A1 - Teaching system, teaching device, robot control device, and program - Google Patents

Abstract

The present invention provides a teaching system for teaching about calibration or position detection, the teaching system comprising: a first device including a first processor and a storage unit storing image data for the calibration or position detection; and a second device including a second processor and a display, wherein the second processor acquires the image data from the first device and displays the image that the image data represents on the display for a visual sensor to capture in performing the calibration or position detection.

Description

Teaching system, teaching device, robot control device, and program

This disclosure relates to a teaching system, a teaching device, a robot control device, and a program.

Various techniques are known in the art for calibrating a camera placed in a robot system using a jig with a specific pattern, such as a dot pattern jig (e.g., Patent Documents 1 and 2). In addition, various techniques are known for capturing an image of a marker with a camera to measure or detect the position of a measurement target (e.g., Patent Documents 3 and 4).

JP 2014-128845 A JP 2019-42834 A JP 2012-218140 A JP 2005-201824 A

When performing calibration or position detection using a dot pattern or marker, the worker generally needs to take out the jig or printed material on which the dot pattern or marker is formed from its storage location to perform the work. There is a demand for technology that can reduce the burden on workers when performing calibration or position detection using a dot pattern or marker.

One aspect of the present disclosure is a teaching system for teaching calibration or position detection, comprising a first device having a first processor and a storage unit that stores image data for the calibration or position detection, and a second device having a second processor and a display. In this teaching system, the second processor acquires the image data from the first device, and displays an image represented by the image data on the display to be captured by a visual sensor in performing the calibration or position detection.

These and other objects, features and advantages of the present invention will become more apparent from the detailed description of exemplary embodiments of the present invention illustrated in the accompanying drawings.

1 is a system configuration diagram of a teaching system for teaching regarding calibration or position detection according to a first embodiment. 1 is a functional block diagram of a teaching system according to a first embodiment. FIG. FIG. 4 is a diagram showing an overall flow of processing related to teaching a predetermined pattern in calibration or position detection in the first embodiment. 3B is a flowchart showing the process of the second device in step S1 of FIG. 3A. FIG. 13 is a diagram showing a state in which a dot pattern for calibration is displayed on the display of the teaching pendant. FIG. 13 is a diagram showing a state in which a dot pattern is displayed in an overlapping manner on an operation screen for teaching. FIG. 13 is a diagram showing an example of an image data registration screen displayed on a display of a teaching pendant. FIG. 11 is a diagram showing the configuration of a device in which a second device that downloads and displays image data of a predetermined pattern is a tablet terminal separate from the teaching pendant. FIG. 2 is a functional block diagram of the tablet terminal. FIG. 13 is a diagram showing a state in which a marker for position detection is displayed on the display of the tablet terminal. FIG. 13 is a diagram showing a state in which a marker for position detection is displayed on a display of a machine tool. FIG. 11 is a system configuration diagram of a teaching system for teaching regarding calibration or position detection according to a second embodiment. FIG. 11 is a functional block diagram of a teaching system according to a second embodiment. 13 is a diagram showing a state in which an image showing a dot pattern and a numerical value of a dot interval as dimensional information is displayed on the display of a teaching pendant. FIG. 13 is a diagram showing a state in which a code image is displayed as dimensional information together with a dot pattern on the display of a teaching pendant. FIG. FIG. 11 is a diagram showing an overall flow of processing related to teaching a predetermined pattern in calibration or position detection in the second embodiment. 14B is a flowchart showing the process performed by the second device in step S1a of FIG. 14A. FIG. 11 is a system configuration diagram of a teaching system for teaching regarding calibration or position detection according to a third embodiment. FIG. 13 is a functional block diagram of a teaching system according to a third embodiment. FIG. 13 is a diagram showing an overall flow of processing related to teaching a predetermined pattern in calibration or position detection in the third embodiment. 17B is a flowchart showing the process performed by the second device in step S1b of FIG. 17A. FIG. 13 is a diagram illustrating an example of an execution screen of a calibration program.

Next, an embodiment of the present disclosure will be described with reference to the drawings. In the drawings, similar components or functional parts are given similar reference symbols. The scale of these drawings has been appropriately changed to facilitate understanding. Furthermore, the form shown in the drawings is one example for implementing the present invention, and the present invention is not limited to the form shown.

In each of the embodiments described below, a teaching system for teaching calibration and position detection will be described.

In this specification, the predetermined pattern for calibration may include various dot patterns and markers known in the art that can be used for calibration. The above-mentioned Patent Documents 1 and 2 describe examples of dot patterns and markers used for calibration. Calibration corresponds to obtaining calibration data (transformation matrix) that associates a position on a camera image with a position in three-dimensional space. The calibration data includes internal parameters corresponding to the transformation from the camera coordinate system to the image coordinate system and external parameters corresponding to the transformation (rotation and translation) from the world coordinate system to the camera coordinate system. As a dot pattern that can be used for calibration, for example, one that satisfies the following requirements is used: (a1) the lattice point interval of the dot pattern is known, (a2) there is a certain number or more of lattice points, and (a3) each lattice point can be uniquely identified. FIG. 4 illustrates such a dot pattern 301. In dot pattern 301, dots are arranged at equal intervals in a grid pattern, and a coordinate system (X axis, Y axis, and origin O) is defined on the dot pattern by large dots M (only some of which are marked with symbols), thereby identifying the position of each grid point.

There are various types of predetermined patterns for detecting the position of a measurement target, called markers or targets, known in the art. Examples of this type of marker are described in the above-mentioned Patent Documents 3 and 4. There are also application examples in which a coordinate system is corrected using a marker. For example, in an application example in which a robot mounted on a cart or AGV (Automated Guided Vehicle) corrects its positional relationship with a machine tool when the robot moves, a marker is attached to one or more locations on the measurement target (machine tool), and the positional relationship is corrected by detecting the marker with a robot equipped with a calibrated camera. Markers that can be used for position detection include not only the marker 302 illustrated in FIG. 5 and those illustrated in Patent Documents 3 and 4, but also various ones that can be detected on an image, such as dot patterns, relatively simple figures, symbols, and characters.

In this specification, the term "predetermined pattern for calibration or position detection" can include the various types described above that can be used for calibration or for detecting or measuring the position of an object.

The teaching system of each embodiment described below includes a first device having a first processor and a storage unit that stores image data of a predetermined pattern for calibration or position detection, and a second device having a second processor and a display. In this teaching system, the second processor of the second device has a function of acquiring image data of the predetermined pattern from the first device, and displaying an image of the predetermined pattern on the display to be captured by a visual sensor when performing calibration or position detection.

First embodiment Fig. 1 shows a system configuration of a teaching system 501 for teaching on calibration or position detection according to a first embodiment. The teaching system 501 includes a robot system 100. Fig. 2 shows a functional block diagram of the teaching system 501. As shown in Fig. 1, the robot system 100 includes a robot 10, a robot control device 20 for controlling the robot 10, a teaching operation panel 30, an image processing device 40 connected to the robot control device 20, and a visual sensor 70. The visual sensor 70 is attached to the tip of the arm of the robot 10 and is connected to the image processing device 40. By calibrating the visual sensor 70, the robot system 100 is able to detect the position of an object using the visual sensor 70 and handle the object.

In the configuration shown in FIG. 1, the robot control device 20 (first device) is responsible for performing calibration or position detection using the visual sensor 70. The teaching operation panel 30 (second device) has the function of acquiring image data of a predetermined pattern for calibration or position detection from the robot control device 20 (first device), and displaying an image of the predetermined pattern based on the acquired image data on the display 32 of the teaching operation panel 30 (second device) so that the image can be captured by the visual sensor.

The robot control device 20 controls the operation of the robot 10 according to the robot program or commands from the teaching operation panel 30. The robot control device 20 may have a hardware configuration as a general computer having a processor 21, memory (ROM, RAM, non-volatile memory, etc.), a storage unit 22, an operation unit, an input/output interface, a network interface, etc. (see Figure 2).

The teaching operation panel 30 is connected to the robot control device 20 by wire or wirelessly, and can provide functions for teaching the robot 10 and inputting various settings. A teaching device configured with various portable terminal devices (tablet terminals, etc.) may be used as the teaching operation panel 30. The teaching operation panel 30 may have a hardware configuration as a general computer having a processor 31, memory (ROM, RAM, non-volatile memory, etc.), a storage device, a display 32, an operation unit 33, an input/output interface, a network interface, etc. (see Figure 2). The operation unit may be configured as a touch operation panel integrated with the display 32.

In Figure 1, the robot 10 is a vertical articulated robot, but various types of robots may be used as the robot 10, such as a horizontal articulated robot, a parallel link robot, or a dual-arm robot, depending on the work target. The robot 10 can perform the desired work using an end effector attached to the wrist. The end effector is an external device that can be replaced depending on the application, such as a hand, a welding gun, or a tool. Figure 1 shows an example in which a hand 11 is used as the end effector.

The image processing device 40 has the function of controlling the visual sensor 70 based on commands from the robot control device 20 (operation control unit 121) and executing image processing (detection, determination, etc.) on the captured images. The visual sensor 70 may be a camera that captures grayscale or color images, or a stereo camera or three-dimensional sensor that can obtain distance images or three-dimensional point clouds.

In this embodiment, the image processing device 40 is disposed as a separate device from the robot control device 20, but the functions of the image processing device 40 may be integrated into the robot control device 20.

As shown in FIG. 2, the robot control device 20 includes an operation control unit 121, a dimension information calculation unit 122, and an image data storage control unit 123. These functional blocks may be realized by the processor 21 of the robot control device 20 executing software. FIG. 2 illustrates a memory unit 22 as a hardware component of the robot control device 20. The memory unit 22 is, for example, a storage device formed of a non-volatile memory or a hard disk device. The memory unit 22 stores various programs such as a robot program for controlling the robot 10, a calibration program, a position detection program (a program for measuring or detecting the position of an object using a predetermined pattern such as a marker), and various setting information. The memory unit 22 also holds image data of a predetermined pattern that can be used for calibration or position detection.

The motion control unit 121 controls the motion of the robot 10 according to the robot program or according to commands from the teaching operation panel 30. The robot control device 20 is equipped with a servo control unit (not shown) that performs servo control on the servo motors of each axis according to commands for each axis generated by the motion control unit 121.

As shown in FIG. 2, the teaching operation panel 30 includes an image data acquisition unit 131, an image data display control unit 132, a brightness adjustment unit 133, and an image data registration unit 134. These functional blocks may be realized by the processor 31 executing software. The image data acquisition unit 131 has a function of acquiring image data of a predetermined pattern for calibration or position detection from the robot control device 20. The image data display control unit 132 has a function of displaying the acquired image of the predetermined pattern on the display 32. With this configuration, the operator can use the teaching operation panel 30 as a calibration tool or a tool for detecting the position of an object by displaying an image of a predetermined pattern such as a dot pattern or marker on the teaching operation panel 30.

In order to perform calibration using a dot pattern, dimensional information on the dot spacing is required. As an example of a configuration for grasping the dimensions of the dot spacing of the dot pattern displayed on the teaching operation panel 30 in the robot control device 20 that performs the calibration, in this embodiment, the robot control device 20 calculates dimensional information based on information related to the display 32 of the teaching operation panel 30, and stores the information in association with image data of the dot pattern.

The dimensional information calculation unit 122 of the robot control device 20 can calculate, for example, from the number of pixels at the lattice point interval in the image data of the dot pattern and information on the resolution and size of the display 32 of the teaching pendant 30, what the dimension of the dot interval will be when the dot pattern is displayed on the display 32. For example, the dimensional information calculation unit 122 may be configured to control the execution of the following procedure.
(Step b1) The input of information (resolution and size) regarding the display 32 is accepted via the teaching pendant 30.
(Step b2) The above-mentioned dimensional information is calculated based on the image data of the dot pattern and information related to the display 32.
(Step b3) The calculated dimensional information is stored in the storage unit 22 in association with the image data of the dot pattern.

The image data storage control unit 123 has a function of associating image data of a dot pattern with dimensional information and storing the associated data in the storage unit 22. When preparing image data of multiple types of dot patterns with different dot spacing, the dimensions of the dot spacing when displayed on the display 32 for each dot pattern (image data) may be calculated and associated with the image data, as shown in Table 1 below. This allows the teaching pendant 30 to be used as multiple types of calibration jigs with different dot spacing.

FIG. 3A shows the overall flow of the process for teaching a specific pattern for calibration or position detection. First, an operator executes a process for acquiring and displaying image data of a specific pattern for calibration or position detection on the teaching operation panel 30 (second device) (step 1).

FIG. 3B is a flowchart showing the process in step S1 in FIG. 3A. In response to an instruction from an operator, the image data acquisition unit 131 of the teaching operation panel 30 (second device) acquires image data of a predetermined pattern for calibration or position detection from the robot control device 20 (first device) (step S11). At this time, if the robot control device 20 holds image data of multiple types of patterns as described above, the image data acquisition unit 131 may be configured to display a list of multiple patterns on the display 32 and accept a user operation to select desired image data from the list.

Next, the image data display control unit 132 of the teaching pendant 30 displays the image of the predetermined pattern acquired from the robot control device 20 on the display 32 (step S12). This makes it possible for the teaching pendant 30 to be used as a jig for calibration or position detection.

As shown in FIG. 3A, the worker then places the teaching pendant 30 (second device) on which the predetermined pattern is displayed at a predetermined position for calibration or position correction where it can be imaged by the visual sensor 70 (step S2). The worker then causes the visual sensor 70 to image the teaching pendant 30 on which the image of the predetermined pattern is displayed, and causes the robot control device 20 to perform calibration or position detection (step S3). In this case, the robot control device 20 (first device) has dimensional information for when the dot pattern is displayed on the teaching pendant 30, and can use that dimensional information when performing calibration.

FIG. 4 shows the state where an image of a dot pattern 301 for calibration is displayed on the display 32 of the teaching operation panel 30 in step S1 above. The dot pattern 301 shown in FIG. 4 satisfies the above-mentioned requirements (a1) to (a3) as a dot pattern for performing calibration. The dot spacing d on the display 32 when the dot pattern 301 is displayed as in FIG. 4 is known to the robot control device 20, which is responsible for performing the calibration. Therefore, the robot control device 20 can appropriately perform the calibration using the value of the dot spacing d held by the robot control device 20. Note that such a dot pattern 301 may be used to detect the position of an object or to set a coordinate system.

The operator can perform calibration or position detection simply by operating the teaching operation panel 30 to download image data of a specified pattern from the robot control device 20 and display it. Therefore, the operator does not need to perform the time-consuming task of retrieving jigs for calibration or position detection from a storage location and setting them up, as was done in the past.

In addition, by storing image data of dot patterns of various sizes in the robot control device 20 as described above, the worker can obtain image data of dot patterns of a size that matches the execution environment from the robot control device 20 at any time, display it on the teaching operation panel 30, and use it for teaching.

According to this embodiment, the operator can perform calibration or position detection by displaying an image of a predetermined pattern on the teaching operation panel 30 whenever necessary. As shown in FIG. 5, while the operation screen 401 for teaching the robot program of the robot 10 is displayed on the teaching operation panel 30, the operator can also activate the function of the image data acquisition unit 131 whenever necessary to download image data of a predetermined pattern from the robot control device 20 and display it overlapping on the operation screen 401 for teaching. The example in FIG. 5 shows a case where a dot pattern 301 is displayed overlapping on the operation screen 401 for teaching.

The brightness adjustment unit 133 of the teaching operation panel 30 has a function of receiving information on the brightness of an image captured by the visual sensor 70 of the teaching operation panel 30 displaying an image of a predetermined pattern from the robot control device 20 (calibration program or position detection program) and adjusting the brightness of the image of the predetermined pattern displayed on the display 32. For example, when the information from the robot control device 20 indicates that the captured image is too dark, the brightness adjustment unit 133 increases the brightness of the image of the predetermined pattern on the display 32. Alternatively, when the information from the robot control device 20 indicates that the captured image is too bright, the brightness adjustment unit 133 decreases the brightness of the image of the predetermined pattern on the display 32. This function makes it possible to automatically adjust the brightness of the predetermined pattern to an appropriate state regardless of the lighting environment in the work space. The operator does not need to perform time-consuming tasks such as adjusting the lighting device in the work space in order to adjust the brightness of the image of the predetermined pattern captured by the visual sensor 70.

The brightness adjustment unit 133 may also have a function for adjusting the brightness of the display of the image of the specified pattern by manual operation. Even in this case, the operator can adjust the brightness of the image of the specified pattern displayed on the teaching operation panel 30 to an appropriate state by looking at the state of the image captured by the visual sensor 70. The operator does not need to carry out time-consuming tasks such as adjusting the lighting equipment in the work space in order to adjust the brightness of the image of the specified pattern captured by the visual sensor 70.

The teaching operation panel 30 (second device), which has the function of displaying an image of a specific pattern, has the function of appropriately adjusting the brightness of the display of the image of the specific pattern in this way, so that the worker does not need to adjust the brightness of the lighting in the work space. Therefore, the burden on the worker when performing calibration or position detection is reduced, and the work can be carried out more efficiently.

The image data registration unit 134 provides a function for registering new image data of a specified pattern for calibration or position detection to the robot control device 20 (first device) that stores image data of a specified pattern. The image data registration unit 134 provides a function for registering any image data among the image data stored in the teaching operation panel 30 (or an external storage device connected to the teaching operation panel 30) in the robot control device 20 as new image data for calibration or position detection.

FIG. 6 shows an example of an image data registration screen 402 displayed on the display 32 of the teaching operation panel 30 by the function of the image data registration unit 134. The registration screen 402 displays a list of image data stored in the teaching operation panel 30 (or an external storage device connected to the teaching operation panel 30). The operator can select the desired image data from the displayed list of image data and press the OK button 411 to register the selected image data in the storage unit 22 of the robot control device 20 as image data of a new pattern. The image data storage control unit 123 of the robot control device 20 provides a function of accepting a registration request from the image data registration unit 134 and storing the new image data in the storage unit 22. Note that any image that can be used for calibration or position detection, such as various marks, figures, symbols, characters, etc., can be used as the image of the newly registered pattern. Note that the image data registration unit 134 may be configured to accept registration of information regarding the dimensions of the pattern when accepting registration of image data of a new pattern. The dimensional information may be, for example, the number of pixels between dots in a dot pattern, or the dimension (millimeters) of the dot spacing on a specific display, as described above. Alternatively, the dimensional information may be information about the size of the marker (for example, the size (number of dots) of circle C, the size (square millimeters) of circle C when it is displayed on a specific display, etc.). This allows the robot control device 20 to hold information that associates image data with dimensional information, such as that shown in Table 1 above, even for new image data.

The above-described embodiment is an example of a configuration in which image data of a predetermined pattern held by the robot control device 20 (first device) is downloaded to the teaching operation panel 30 (second device) and displayed, but the second device that downloads and displays the image data of the predetermined pattern is not limited to the teaching operation panel 30. For example, the configuration may be such that the image data of the predetermined pattern is downloaded and displayed by a portable terminal device separate from the teaching operation panel 30. In this case, the terminal device can be positioned as a teaching device for performing calibration or position detection-related teaching.

FIG. 7 shows an example of the equipment configuration of such a teaching system 501A. In the teaching system 501A, a tablet terminal 80 separate from the teaching pendant 30 downloads and displays image data of a predetermined pattern. FIG. 8 shows a functional block diagram of the tablet terminal 80 in this system configuration. As shown in FIG. 8, the tablet terminal 80 includes an image data acquisition unit 181, an image data display control unit 182, and an image data registration unit 184. The functions of the image data acquisition unit 181, the image data display control unit 182, and the image data registration unit 184 are equivalent to the functions of the image data acquisition unit 131, the image data display control unit 132, and the image data registration unit 134 of the teaching pendant 30 described above. Note that FIG. 8 also shows a display 82 and an operation unit 83 as hardware components of the tablet terminal 80.

The tablet terminal 80 may have a hardware configuration as a general computer having a processor 81, memory (ROM, RAM, non-volatile memory, etc.), a storage device, a display 82, an operation unit 83, an input/output interface, a network interface, etc. The operation unit may be configured as a touch operation panel integrated with the display.

The tablet terminal 80 may be connected to the robot control device 20 so as to be able to communicate with it, or it may not be able to communicate with it. If the tablet terminal 80 is not able to communicate with the robot control device 20, for example, the worker may download and store image data of the robot control device 20 in a USB memory. The worker may then connect this USB memory to the tablet terminal 80, and obtain and display image data of a desired pattern on the tablet terminal 80.

In this case, the robot control device 20 may store information about the display 82 of the tablet terminal 80 in addition to information about the display 32 of the teaching operation panel 30. This allows the robot control device 20 to calculate and store the size of the dot spacing of the dot pattern on the display 82 from the number of pixels of the dot spacing in the image data of the dot pattern and the resolution and size of the display 82 of the tablet terminal 80. This allows the robot control device 20 to store dimensional information of the dot spacing when the dot pattern is displayed on the teaching operation panel 30 and dimensional information of the dot spacing when the dot pattern is displayed on the tablet terminal 80, in association with the image data, as shown in Table 2 below.

The information shown in Table 2 can also be expanded to accommodate three or more types of terminal devices. In this case, the dot spacing when the dot pattern is displayed on three or more types of terminal devices can be calculated based on the number of pixels for the dot spacing in the dot pattern and the display information (resolution, size) of the three or more types of terminal devices.

In this manner, when the robot control device 20 is configured to hold dimensional information for multiple types of displays, the robot control device 20 (processor 21) may be configured to obtain information about which display is to be used in performing calibration or position detection through user input, for example, via the teaching operation panel 30.

Even in the configuration example shown in Figures 7 and 8, the dimensions of the dot spacing when the dot pattern is displayed on the tablet terminal 80 are known to the robot control device 20, so the same advantages as those of the embodiment described above with reference to Figures 1 to 6 can be obtained. Note that, in the configuration example shown in Figures 7 and 8, the advantage is that the operation of the robot 10 using the teaching operation panel 30 and calibration or position detection using the tablet terminal 80 can be performed simultaneously in parallel.

9 shows a state in which an image of the marker 302 acquired from the robot control device 20 is displayed on the display 82 of the tablet terminal 80. As an example, the marker 302 is a type of marker that includes mutually perpendicular straight lines a and b and a circle C of a known size. As an example, the marker 302 is used as follows. When detecting the position of an object using the marker 302, the worker calibrates the visual sensor 70 in advance. Then, the worker operates the tablet terminal 80 to download image data of the marker 302 from the robot control device 20 and display it on the display 82 (step S1). The worker places the tablet terminal 80 on which the marker 302 is displayed at a predetermined position of the measurement object (workbench, machine tool, etc.) (step S2), and causes the robot control device 20 to execute a position detection program (step S3). The position detection program grasps the three-dimensional position of the marker 302 based on the detected position of the marker 302 on the captured image, and obtains the three-dimensional position of the measurement object. A stereo measurement method using a visual sensor 70 may be used to measure the three-dimensional position of the marker 302. Therefore, even when performing such position detection, the worker does not need to take out a marker jig or a printed copy of the marker stored in a separate location and install it on the target.

When detecting the position of an object using a marker of a type such as marker 302, if dimensional information (such as the size of circle C) is required for teaching the marker, the robot control device 20 (first device) may associate the dimensional information with the image data of the marker and store the dimensional information when the marker is displayed on the display 82 of the tablet terminal 80 (second device), and use this dimensional information when detecting the position of the object.

The second device that displays an image of the predetermined pattern may be a machine tool used together with the robot 10. FIG. 10 shows a schematic diagram of the equipment configuration in this case. For example, this case corresponds to an application example in which the robot 10 is mounted on a cart or AGV and placed at a predetermined position relative to the machine tool 90 to load and unload workpieces from the machine tool 90. The machine tool 90 incorporates a control device (numerical control device) 91 with an integrated display 92. The control device 91 can download and display image data of the predetermined pattern from the robot control device 20, for example, via a network. Alternatively, image data of the predetermined pattern may be registered in advance in the memory unit of the control device 91.

10 shows a state in which the marker 302 is displayed on the display 92 of the control device 91. The display 92 of the control device 91 is attached to a fixed position on the machine tool 90, so that the marker 302 can be used as an indicator of the position of the machine tool 90 by displaying the marker 302 at a predetermined position on the display 92. In this case, for example, when performing position detection, the worker operates the control device 91 to display the marker 302 on the display 92. The robot control device 20 measures the positional relationship between the robot 10 and the machine tool 90 by capturing an image of the marker 302 displayed on the display 92 with the visual sensor 70. In this case, the robot control device 20 can hold information regarding the specifications of the display 92 of the control device 91 (resolution, screen size, etc.) and use the dimensional information of the marker 302 when displayed on the display 92 based on the image data of the marker 302 for detection. In this configuration, the processor of the control device 91 can execute the functions of an image data acquisition unit 181, an image data display control unit 182, and an image data registration unit 184, as described above with reference to FIG. 8 for the tablet terminal 80.

Second embodiment FIG. 11 is a diagram showing the equipment configuration of a teaching system 502 according to the second embodiment. The teaching system 503 includes a robot system 100A. In the teaching system 502 shown in FIG. 11, the second device that displays an image of a predetermined pattern for calibration or position detection is the teaching pendant 30, and the first device that provides image data to the second device (teaching pendant 30) is an external device other than the robot control device 20A. In this embodiment, as an example, the external device is another control device (numerical control device, robot control device, etc.) 220 arranged in the factory where the robot system 100A is arranged. It is assumed that the teaching pendant 30 can communicate with the control device 220 by wire or wirelessly via a network in the factory.

The control device 220 may have a hardware configuration as a general computer having a processor 224, memory (ROM, RAM, non-volatile memory, etc.), a storage unit 225, a display unit, an operation unit, an input/output interface, a network interface, etc. (see FIG. 12).

In this embodiment, the control device 220 (first device) holds image data of a predetermined pattern for calibration or position detection in a format similar to that of the robot control device 20 in the first embodiment. Therefore, with regard to a dot pattern, the control device 220 holds dimensional information of the dot spacing when the dot pattern is displayed on the display 32 of the teaching operation panel 30 in association with the image data of the dot pattern. In this embodiment, in order to provide the dimensional information of the dot spacing to the robot control device 20A that is responsible for calibration or position detection, an image showing the dimensional information is displayed on the display 32 of the teaching operation panel 30 together with an image of the dot pattern.

12 shows a functional block diagram of the robot control device 20A, teaching operation panel 30, and control device 220 according to the second embodiment. The control device 220 includes a dimension information calculation unit 221 and an image data storage control unit 222. The dimension information calculation unit 221 and the image data storage control unit 222 have functions equivalent to the dimension information calculation unit 122 and the image data storage control unit 123 of the robot control device 20 of the first embodiment. That is, the dimension information calculation unit 221 and the image data storage control unit 222 can execute the same processes as the above-mentioned (steps b1) to (steps b3) in advance and store image data and dimension information of a predetermined pattern in the storage unit 225. The storage unit 225 is a storage device configured with a non-volatile memory or a hard disk, etc., and stores image data and dimension information of a predetermined pattern in addition to programs and various setting information related to the control of the machine in the control device 220.

The image data acquisition unit 131 of the teaching operation panel 30 has a function of downloading image data and dimensional information of a specified pattern from the memory unit 225 of the control device 220. The image data display control unit 132 and brightness adjustment unit 133 have the same functions as those described above in relation to the first embodiment. The image data registration unit 134 provides the control device 220 with a function for newly registering image data of a specified pattern for calibration or position detection.

The robot control device 20A includes an operation control unit 121 and a dimension recognition unit 124. The storage unit 22 stores a robot program, a calibration program, a position detection program, and various other setting information. The dimension recognition unit 124 provides a function for recognizing dimensions from an image of dimensional information displayed together with an image of a specified pattern.

FIGS. 13A and 13B show examples of images showing dimensional information about dot spacing together with a dot pattern 301 displayed on the display 32 of the teaching pendant 30. FIG. 13A shows an example in which an image G1 showing the numerical value of the dot spacing is displayed as dimensional information. FIG. 13B shows an example in which an image G2 in which the numerical value of the dot spacing is coded as dimensional information is displayed. The dimensional information calculation unit 221 may have a function for coding the calculated dot spacing dimension as in image G2. The control device 220 may store such image G1 or image G2 as dimensional information.

FIG. 14A shows the overall flow of the process related to teaching a specified pattern for calibration or position detection. Here, the flow is explained focusing on the process when a dot pattern is used as the specified pattern. First, the operator executes a process for acquiring and displaying image data and dimensional information of the specified pattern for calibration or position detection on the teaching operation panel 30 (second device) (step S1a).

FIG. 14B is a flowchart showing the processing content in step S1a of FIG. 14A. In response to instructions from an operator, the image data acquisition unit 131 of the teaching pendant 30 (second device) acquires image data of a predetermined pattern for calibration or position detection, and dimensional information from the control device 220 (first device) (step S11a). At this time, if the control device 220 holds image data of multiple types of patterns, the image data acquisition unit 131 may be configured to display a list of multiple patterns on the display 32 and accept a user operation to select desired image data from the list.

Next, the image data display control unit 132 of the teaching pendant 30 causes the image and dimensional information of the predetermined pattern acquired from the control device 220 to be displayed on the display 32 (step S12a). This makes it possible for the teaching pendant 30 to be used as a jig for calibration or position detection.

14A, the operator then places the teaching pendant 30, on which an image showing dimensional information is displayed together with an image of a dot pattern, at a predetermined position for calibration and position correction where an image can be captured by the visual sensor 70 (step S2). Next, the operator causes the robot control device 20A to recognize the dimensional information of the dot interval (step S2a). In this embodiment, the operator can cause the robot control device 20A to recognize the dot interval by any of the following methods (c1) to (c3).
(c1) An operator directly inputs the data into the setting items of the calibration program.
(c2) The robot control device reads and recognizes the numerical values on the image.
(c3) The robot control device reads and recognizes the code information on the image.

The above method (c1) is an effective method when an image G1 showing the numerical value of the dot spacing is displayed together with an image of a dot pattern, as shown in FIG. 13A. In this case, the worker can see the image of the dimensional information displayed together with the image of the dot pattern, grasp the dot spacing, and directly input it into the setting item of the calibration program. FIG. 18 shows an example of an execution screen 450 of the calibration program. The execution screen 450 displays an input field 451 for specifying the dot spacing together with an image G5 captured by the visual sensor 70 of the teaching pendant 30 placed in a predetermined position. The worker can input the dot spacing by directly entering a numerical value into the input field 451, or by specifying the dot spacing from a menu list of numerical values. The worker can then perform a predetermined operation on the execution screen 450 to execute the calibration and check the results.

Note that when the above method (c1) is applied as a method for making the robot control device 20A know the dot spacing, the robot control device 20A does not need to have the functionality of the dimension recognition unit 124.

The above method (c2) is an effective method when displaying an image G1 showing the numerical value of the dot spacing together with an image of a dot pattern, as shown in FIG. 13A. The dimension recognition unit 124 of the robot control device 20A has the function of recognizing the numbers of the dimensional information from an image captured by the visual sensor 70 of the teaching operation panel 30 in a state in which the image G1 of the dimensional information is displayed on the display 32 together with the dot pattern, and providing the numbers to the calibration program. Various character recognition techniques known in the field can be used to recognize the numbers from the image.

The above method (c3) is an effective method when displaying an image G2 of a code representing the dot spacing together with an image of a dot pattern, as shown in FIG. 13B. The dimension recognition unit 124 of the robot control device 20A has a function of reading the code from an image captured by the visual sensor 70 of the teaching operation panel 30 in a state in which an image G2 of dimensional information is displayed on the display 32 together with a dot pattern, and providing the code to the calibration program. The code may be a one-dimensional code or a two-dimensional code. Various code recognition technologies known in the field can be used to recognize the code.

Next, the worker causes the visual sensor 70 to capture an image of the dot pattern and causes the robot control device 20A to perform calibration or position detection (step S3).

In the device configuration of this embodiment, when a marker other than a dot pattern is used as the predetermined pattern to be displayed on the teaching operation panel 30 (second device), in an application example in which the acquisition and display of dimensional information on the teaching operation panel 30 (second device) is not required, the process flow shown in FIG. 3A-FIG. 3B in the first embodiment can be applied instead of the process flow of FIG. 14A-FIG. 14B described above. In addition, when teaching of a marker is required in the position detection program, the control device 220 (first device) holds configuration information (dimensional information, etc.) of the marker together with the image data of the marker. Then, the teaching operation panel 30 (second device) may acquire the configuration information together with the image data of the marker from the control device 220 (first device), and provide the acquired configuration information to the robot control device 20A so that it can be used for teaching the marker. Alternatively, the teaching operation panel 30 may display an image showing the dimensional information of the marker on the display 32 together with the image of the marker. In this case, the robot control device 20A (dimension recognition unit 124) can recognize the dimensional information from the captured image in which the visual sensor 70 captures an image showing the marker and the dimensional information.

According to this embodiment, the operator can perform calibration or position detection by displaying an image of a specific pattern on the teaching operation panel 30 whenever necessary. The operator does not need to perform the time-consuming task of retrieving a jig for calibration or position detection from a storage location and setting it up, as was done in the past. By displaying an image showing dimensional information together with an image of the specific pattern, it is also possible to have the robot control device 20A recognize the dimensional information, so the operator can perform calibration simply by operating the teaching operation panel 30 to download image data of the specific pattern from the control device 220 and display it.

In the example of the device configuration shown in FIG. 11, the second device that downloads and displays the image data of the specified pattern is the teaching operation panel 30, but a portable terminal device other than the teaching operation panel 30 may be used as the second device. In this case, the terminal device is positioned as a teaching device for teaching regarding calibration or position detection. In this case, the control device 220 associates the dimensional information of each dot interval when the image of the dot pattern is displayed on each terminal device, as described in Table 2 of the first embodiment, with the image data of the dot pattern and stores it. When a request for downloading image data is made from a terminal device, the control device 220 (processor 224) may be configured to identify the terminal device that requested the image data and provide the terminal device with the dimensional information corresponding to the terminal device along with the image data of the specified pattern. In addition, the second device that downloads and displays the image data of the specified pattern may be a machine tool.

With this configuration, images of a specific pattern can be displayed on various terminal devices other than the teaching pendant 30, making them available for calibration and position detection.

Note that while FIG. 11 shows an example in which the first device that provides image data of a predetermined pattern is the machine control device 220, the first device that provides image data of a predetermined pattern can be various external devices. For example, the first device that provides image data of a predetermined pattern can be the teaching operation panel 30 used by the worker, or a computer or cloud that is network-connected to another terminal device.

In this embodiment, the configuration has been described where the teaching operation panel 30 displays the dimensional information received from the control device 220 together with an image of a specified pattern, but the teaching operation panel 30 may also transmit the dimensional information received from the control device 220 to the robot control device 20A so that the robot control device 20A can use it for calibration and position detection.

The first and second embodiments described above are configuration examples in which a first device providing image data of a predetermined pattern has a function for generating dimensional information of the dot interval when the dot pattern is displayed on a second device based on information about the display of the second device displaying the image data of the predetermined pattern. In this embodiment, the second device displaying the image data of the predetermined pattern has a function for generating dimensional information of the dot interval.

FIG. 15 is a diagram showing the equipment configuration of a teaching system 503 according to the third embodiment. The teaching system 503 includes a robot system 100B. As shown in FIG. 15, in the teaching system 503, the second device that displays an image of a predetermined pattern for calibration or position detection is a teaching operation panel 30B, and the first device that provides image data to the second device (teaching operation panel 30B) is an external device 320 that is network-connected to the teaching operation panel 30B. In this case, the network may include an in-house network such as a LAN (Local Area Network) or a commercial network such as the Internet.

The external device 320 may include various devices and computer systems, such as a control device arranged in the same factory as the robot system 100B, a computer, a server connected via a commercial network, and a cloud. The external device 320 may have a hardware configuration as a general computer having a processor 321, memory (ROM, RAM, non-volatile memory, etc.), a storage unit 322, a display unit, an operation unit, an input/output interface, a network interface, etc. (see FIG. 16).

FIG. 16 shows a functional block diagram of the robot control device 20A, teaching operation panel 30B, and external device 320 in the teaching system 503 according to the third embodiment. The external device 320 holds image data of a predetermined pattern for calibration or position detection in the memory unit 322.

The teaching operation panel 30B has the functions of an image data acquisition unit 131, an image data display control unit 132, a brightness adjustment unit 133, and an image data registration unit 134. These functions have been described above, so details will be omitted. The image data acquisition unit 131 has the function of acquiring image data of a specified pattern from the external device 320. The image data registration unit 134 provides a function of registering new image data that can be used for calibration or position detection in the external device 320. The teaching operation panel 30B according to this embodiment further includes a dimension information generation unit 135.

The dimensional information generating unit 135 has a function of calculating, based on the resolution and size information of the display 32, the size of the dot spacing of the dot pattern when the image data of the dot pattern downloaded from the external device 320 is displayed on the display 32. Specifically, the dimensional information generating unit 135 analyzes the image of the dot pattern and determines the number of pixels of the dot spacing. Then, based on the resolution and size of the display 32, the dimensional information generating unit 135 calculates the size of the dot spacing on the display 32 and generates dimensional information. The dimensional information is, for example, a numerical value or a code.

The image data display control unit 132 can display an image showing the dimensional information on the display 32 together with the image of the dot pattern. Examples of the display format in this case include displaying an image of numerical values showing the dimensional information together with the image of the dot pattern, as shown in Figures 13A and 13B, and displaying an image in which the dimensional information is coded together with the image of the dot pattern.

As shown in FIG. 16, the robot control device 20A may have the same functional configuration as the robot control device 20A in the second embodiment described above.

FIG. 17A shows the overall flow of processing related to teaching a specified pattern for calibration or position detection. Here, the flow is explained focusing on the processing when a dot pattern is used as the specified pattern. First, the operator acquires image data of the specified pattern for calibration or position detection on the teaching operation panel 30B (second device) and executes processing to display it together with dimensional information (step S1b).

FIG. 17B is a flow chart showing the processing content in step S1b of FIG. 17A. In response to an instruction from an operator, the image data acquisition unit 131 of the teaching pendant 30B (second device) acquires image data of a predetermined pattern for calibration or position detection from the external device 320 (first device) (step S11). The processor 321 of the external device 320 (first device) has a function of sending image data in the memory unit 322 to the teaching pendant 30B in response to a request from the teaching pendant 30B. At this time, if the external device 320 holds image data of multiple types of patterns, the image data acquisition unit 131 may be configured to display a list of multiple patterns on the display 32 and accept a user operation to select desired image data from the list.

Then, the dimensional information generating unit 135 of the teaching pendant 30B (second device) calculates dimensional information of the dot spacing when the dot pattern is displayed on the display 32 based on the number of pixels between dots in the acquired image data of the dot pattern and the resolution and size information of the display 32 (step S11b). Then, the image data display control unit 132 causes the display 32 to display an image representing the dimensional information together with the image of the dot pattern (step S12a).

As shown in FIG. 17A, the operator then places the teaching pendant 30B, on which an image showing dimensional information is displayed together with an image of a dot pattern, at a predetermined position for calibration and position correction where an image can be captured by the visual sensor 70 (step S2). Next, the operator causes the robot control device 20A to recognize the dimensional information of the dot spacing (step S2a). Here, the operator may cause the robot control device 20A to recognize the dot spacing using any of the methods (c1) to (c3) described in the second embodiment.

Then, the worker causes the visual sensor 70 to capture an image of the dot pattern and perform calibration or position detection (step S3).

In the case of using a marker other than a dot pattern as the predetermined pattern to be displayed on the teaching operation panel 30B (second device) in the equipment configuration of this embodiment, in an application example in which the generation and display of dimensional information on the teaching operation panel 30B (second device) is not required, the process flow shown in Figures 3A-3B in the first embodiment can be applied instead of the process flow of Figures 17A-17B described above. In addition, when teaching of a marker is required in the position detection program, the external device 320 (first device) stores configuration information of the marker together with image data of the marker. Then, the teaching operation panel 30B (second device) may obtain the configuration information together with image data of the marker from the external device 320 (first device), and provide the obtained configuration information to the robot control device 20A so that it can be used for teaching of the marker.

In this way, in this embodiment, the second device that displays the image of the specified pattern is configured to calculate and display the dot spacing of the dot pattern, so there is no need to generate and store dimensional information in advance in the first device that provides the image data of the specified pattern.

According to this embodiment, the operator can perform calibration or position detection by displaying an image of a specific pattern on the teaching operation panel 30B whenever necessary. With the configuration of this embodiment, the operator does not need to perform the time-consuming task of retrieving a jig for calibration or position detection from a storage location and setting it up, as was done in the past. By displaying an image showing dimensional information together with an image of the specific pattern, it is also possible to have the robot control device 20A recognize the dimensional information, so the operator can perform calibration simply by operating the teaching operation panel 30B to download image data of the specific pattern from the external device 320 and display it.

In the configuration example shown in FIG. 15, the second device that downloads and displays an image of a predetermined pattern from the first device (external device 320) is the teaching operation panel 30B, but this is an example, and the second device that downloads an image of a predetermined pattern from the first device (external device 320), generates dimensional information, and displays it together with the image of the predetermined pattern may be a portable terminal device separate from the teaching operation panel 30B. In this case, the terminal device is positioned as a teaching device for performing calibration or position detection-related teaching.

In this embodiment, the configuration has been described where the teaching operation panel 30B displays the calculated dimensional information together with an image of a specified pattern, but the teaching operation panel 30B may also transmit the calculated dimensional information to the robot control device 20A so that the robot control device 20A can use it for calibration and position detection.

As described above, according to each embodiment, the burden on the worker when performing calibration or position detection using a specified pattern is reduced, and these tasks can be performed efficiently.

The functional distribution shown in the functional block diagrams of the above-mentioned embodiments is an example, and various modifications of the functional distribution are possible. For example, the teaching operation panel 30 shown in the first embodiment does not need to have the functions of the brightness adjustment unit 133 and the image data registration unit 134.

As a variation of the above-described embodiment, a configuration example is also possible in which a first device that has the function of providing image data also has an application program that displays image data along with image data of a predetermined pattern, and provides such an application along with the image data to a second device that has the function of displaying the predetermined pattern. This application program may have a function of obtaining display information (resolution, size, etc.) of the device that executes this application, and calculating dimensional information when the predetermined pattern is displayed. The second device that executes this application may display an image (an image of numerical values or codes) that represents the calculated dimensional information together with an image of the pattern, as shown in Figures 13A-13B. Alternatively, the second device that executes this application may provide the calculated dimensional information to the first device.

This configuration will be described below with reference to the device configuration in FIG. 7 as an example. The robot control device 20, which serves as a first device, has image data of a predetermined pattern together with an application program for displaying the image data. The robot control device 20 transmits the application program together with the image data to the tablet terminal 80, which serves as a second device. The tablet terminal 80 executes the application program, displays an image of the predetermined pattern, and calculates dimensional information. The tablet terminal 80 may display an image (numerical values or codes) representing the dimensional information together with the image of the predetermined pattern. Alternatively, the tablet terminal 80 may transmit the calculated dimensional information to the robot control device 20. The robot control device 20 can use the dimensional information recognized from the image captured by the visual sensor 70 or provided by the tablet terminal 80 for calibration or position detection.

In the above-described embodiments, the functional blocks in the functional block diagrams relating to the robot control device, teaching operation panel, tablet terminal, and external device may be realized by one or more processors of these devices executing various software stored in a storage device, or may be realized by a configuration mainly based on hardware such as an ASIC (Application Specific Integrated Circuit).

The programs for executing the teaching processes for calibration or position detection in the above-mentioned embodiments can be recorded on various computer-readable recording media (e.g., semiconductor memories such as ROM, EEPROM, and flash memory, magnetic recording media, and optical disks such as CD-ROM and DVD-ROM).

Although the present disclosure has been described in detail, the present disclosure is not limited to the individual embodiments described above. Various additions, substitutions, modifications, partial deletions, etc. are possible to these embodiments without departing from the gist of the present disclosure, or without departing from the spirit of the present disclosure derived from the contents described in the claims and their equivalents. These embodiments can also be implemented in combination. For example, in the above-mentioned embodiments, the order of each operation and the order of each process are shown as examples, and are not limited to these. The same applies when numerical values or formulas are used to explain the above-mentioned embodiments.

The following additional notes are provided regarding the above embodiment and modifications.
(Appendix 1)
A teaching system (501, 501A, 502, 503) for teaching on calibration or position detection,
A first processor;
A first device (20, 220, 320) having a storage unit that stores image data for the calibration or position detection;
A second processor; and
a second device (30, 80, 30B) having a display;
The second processor of the second device
acquiring the image data from the first device;
displaying an image represented by the image data on the display so as to be captured by a visual sensor in performing the calibration or position detection;
Teaching system (501, 501A, 502, 503).
(Appendix 2)
The first device is a robot control device (20) that performs the calibration or position detection,
The first processor of the first device
performing the calibration or position detection based on a captured image obtained by capturing the image displayed on the display of the second device by the visual sensor;
The teaching system (501, 501A) described in Appendix 1, wherein the second device is a teaching operation panel (30) connected by wire or wirelessly to the robot control device (20), or a device (80) separate from the teaching operation panel.
(Appendix 3)
The storage unit of the first device (20) stores information about a size of the image on a display of the second device when the image represented by the image data is displayed on the display of the second device in association with the image data;
The teaching system (501, 501A) of Appendix 2, wherein the first processor of the first device uses information about the dimensions in performing the calibration or the position detection.
(Appendix 4)
A robot control device (20A) having a third processor that executes the calibration or the position detection,
the first device is a device (220, 320) separate from the robot control device;
The teaching system described in Appendix 1, wherein the second device is a teaching operation panel (30, 30B) connected by wire or wirelessly to the robot control device (20A), or a device separate from the teaching operation panel.
(Appendix 5)
The storage unit of the first device (220) holds information about a size of an image represented by the image data when the image is displayed on the display of the second device (30) in association with the image data;
The second processor receives information regarding the dimensions together with the image data from the first device (220);
The second processor,
(1) displaying, on the display, an image showing information related to the dimensions together with the image represented by the image data;
(2) transmitting information regarding the dimensions to the robot controller;
The teaching system (502) according to claim 4, which executes one of the processes.
(Appendix 6)
The second processor calculates information regarding dimensions of the image based on the image data when the image is displayed on the display based on the acquired image data and information regarding a resolution and a size of the display;
The second processor,
(1) displaying, on the display, an image showing information related to the dimensions together with the image represented by the image data;
(2) transmitting information about the calculated dimensions to the robot controller;
The teaching system (503) according to appendix 4, which executes any one of the processes above.
(Appendix 7)
The teaching system (502, 503) of appendix 5 or 6, wherein the other device is either a machine control device, an external device, or a cloud.
(Appendix 8)
The teaching system (502, 503) of any one of appendix 5 to 7, wherein the image indicating the dimensional information is an image of a numerical value of the dimension or an image in which the dimension is coded.
(Appendix 9)
The second processor displays on the display an image showing information related to the dimensions together with an image represented by the image data;
The third processor of the robot control device (20A)
Recognizing information regarding the size from a captured image obtained by capturing an image represented by the image data displayed on the display of the second device (30, 30B) and an image showing information regarding the size by the visual sensor;
A teaching system (502, 503) according to any one of appendixes 5 to 8, wherein information about the recognized dimensions is used in performing the calibration or position detection.
(Appendix 10)
The teaching system (501501A, 502) described in Appendix 3 or 5, wherein the first processor of the first device (20, 220) generates information regarding the dimensions based on image data of the dot pattern and information regarding the resolution and size of the display of the second device.
(Appendix 11)
The second processor of the second device (30, 80, 30B)
A teaching system (501, 501A, 502, 503) described in any one of appendices 1 to 10, which registers new image data of a predetermined pattern that can be used for the calibration or position detection and information regarding the dimensions of the new image data in the memory unit of the first device based on a user operation.
(Appendix 12)
The second device is a teaching pendant (30, 30B) connected to a robot control device,
The second processor of the second device obtains information regarding the brightness of an image captured by the visual sensor of the second device displaying an image represented by the image data on the display from the robot control device, and adjusts the display brightness of the display displaying the image based on the information regarding the brightness.
(Appendix 13)
the storage unit of the first device further stores an application program for displaying an image represented by the image data;
The second processor obtains the application program together with the image data from the first device;
The teaching system (501, 501A, 502) described in Appendix 1, wherein the second processor executes the application program to display an image represented by the image data on a display of the first device.
(Appendix 14)
the application program further has a function of calculating information regarding dimensions when an image represented by the image data is displayed on a display of a device that executes the application, based on information regarding the display of the device;
The second processor,
(1) displaying an image showing the information about the dimensions obtained by executing the application together with an image represented by the image data;
(2) transmitting information regarding the dimensions to the first device;
The teaching system (501, 501A, 502) according to Appendix 13, which executes any one of the processes above.
(Appendix 15)
A teaching device (30, 80, 30B) used for teaching regarding calibration or position detection,
A display (32, 82);
A processor (31, 81),
The processor (31, 81)
Acquiring image data for the calibration or position detection from an external device;
A teaching device (30, 80, 30B) displays an image represented by the image data on the display to be captured by a visual sensor in performing the calibration or position detection.
(Appendix 16)
The processor further acquires, from the external device, information regarding a size of the image represented by the image data when the image is displayed on the display;
The processor,
(1) displaying, on the display, an image showing information related to the dimensions together with the image represented by the image data;
(2) transmitting information about the dimensions to a robot control device that controls the calibration or position detection;
The teaching device (30) according to claim 15, which executes any one of the processes above.
(Appendix 17)
The processor calculates information about dimensions of the image based on the image data when the image is displayed on the display based on the acquired image data and information about a resolution and a size of the display;
The processor,
(1) displaying, on the display, an image showing information related to the dimensions together with the image represented by the image data;
(2) transmitting information about the calculated dimensions to a robot control device that controls the calibration or position detection;
A teaching device (30B) according to appendix 15, which executes any one of the processes above.
(Appendix 18)
The processor (31, 81) further
A teaching device (30, 80, 30B) described in any one of appendices 15 to 17, which registers new image data of a predetermined pattern that can be used for the calibration or position detection and information regarding the dimensions of the new image data in a memory unit of the external device based on a user operation.
(Appendix 19)
The external device is a robot control device (20, 20A) that performs the calibration or position detection,
The processor (31) obtains information regarding the brightness of an image captured by the visual sensor of the teaching device displaying an image represented by the image data on the display from the robot control device, and adjusts the display brightness of the display showing the image based on the information regarding the brightness.
(Appendix 20)
A robot control device (20, 20A),
A processor (21);
A storage unit (22) that stores image data for calibration or position detection,
The processor (21)
Transmitting the image data to an external device in response to a request from the external device;
A robot control device (20, 20A) that executes the calibration or position detection based on a captured image obtained by capturing, by a visual sensor, an image represented by the image data displayed on a display of the external device.
(Appendix 21)
the storage unit stores information about a size of the image on a display of the external device when the image represented by the image data is displayed on the display of the external device in association with the image data;
21. The robot control device (20) of claim 20, wherein the processor uses information about the dimensions in performing the calibration or the position detection.
(Appendix 22)
The processor (21)
A robot control device (20) as described in Appendix 19, which calculates information regarding the dimensions based on the image data and information regarding the resolution and size of the display of the external device.
(Appendix 23)
an image represented by the image data is displayed on a display of the external device together with information regarding a size of the image when the image is displayed on the display;
The processor (21)
Recognizing information regarding the size from a captured image obtained by capturing an image represented by the image data and an image represented by the information regarding the size displayed on the display of the external device by the visual sensor;
A robot control device (20A) as described in Appendix 20, wherein information regarding the recognized dimensions is used in performing the calibration or position detection.
(Appendix 24)
The computer processor
A procedure for storing image data for calibration or position detection in a storage unit;
transmitting the image data to an external device in response to a request from the external device;
A procedure for performing the calibration or position detection based on an image captured by a visual sensor of an image represented by the image data displayed on a display of the external device, and a program for causing the procedure to execute the procedure.

REFERENCE SIGNS LIST 10 Robot 11 Hand 20, 20A Robot control device 21 Processor 22 Memory unit 30, 30B Teaching operation panel 31 Processor 32 Display 33 Operation unit 40 Image processing device 70 Visual sensor 80 Tablet terminal 81 Processor 82 Display 83 Operation unit 90 Machine tool 91 Control device 92 Display 100, 100A, 100B Robot system 121 Operation control unit 122 Dimension information calculation unit 123 Image data storage control unit 124 Dimension recognition unit 131 Image data acquisition unit 132 Image data display control unit 133 Brightness adjustment unit 134 Image data registration unit 135 Dimension information generation unit 181 Image data acquisition unit 182 Image data display control unit 184 Image data registration unit 220 Control device 221 Dimension information calculation unit 222 Image data storage control unit 224 Processor 225 Storage unit 301 Dot pattern 302 Marker 320 External device 321 Processor 322 Storage unit 402 Registration screen 501, 501A, 502, 503 Teaching system

Claims (24)

A teaching system for teaching calibration or position detection, comprising:
A first processor;
A first device having a storage unit that stores image data for the calibration or position detection;
A second processor; and
a second device having a display;
The second processor,
acquiring the image data from the first device;
displaying an image represented by the image data on the display so as to be captured by a visual sensor in performing the calibration or position detection;
Teaching system. the first device is a robot control device that performs the calibration or position detection;
The first processor of the first device
performing the calibration or position detection based on a captured image obtained by capturing the image displayed on the display of the second device by the visual sensor;
2. The teaching system according to claim 1, wherein the second device is a teaching pendant connected to the robot control device by wire or wirelessly, or is a device separate from the teaching pendant. the storage unit of the first device stores information about a size of the image on a display of the second device when the image represented by the image data is displayed on the display of the second device in association with the image data;
The teaching system of claim 2 , wherein the first processor of the first device uses information about the dimensions in performing the calibration or the position detection. a robot control device having a third processor for performing the calibration or the position detection;
the first device is a device separate from the robot control device,
2. The teaching system according to claim 1, wherein the second device is a teaching pendant connected to the robot control device by wire or wirelessly, or is a device separate from the teaching pendant. the storage unit of the first device holds information relating to a size of an image represented by the image data when the image is displayed on the display of the second device in association with the image data;
The second processor receives information regarding the dimensions along with the image data from the first device;
The second processor,
(1) displaying, on the display, an image showing information related to the dimensions together with the image represented by the image data;
(2) transmitting information regarding the dimensions to the robot controller;
The teaching system according to claim 4 , wherein the teaching system executes any one of the processes above. The second processor calculates information regarding dimensions of the image based on the image data when the image is displayed on the display based on the acquired image data and information regarding a resolution and a size of the display;
The second processor,
(1) displaying, on the display, an image showing information related to the dimensions together with the image represented by the image data;
(2) transmitting information about the calculated dimensions to the robot controller;
5. The teaching system according to claim 4, wherein the teaching system executes any one of the processes above. The teaching system according to claim 5 or 6, wherein the other device is either a machine control device, an external device, or a cloud. The teaching system according to any one of claims 5 to 7, wherein the image showing information about the dimension is an image of the numerical value of the dimension or an image in which the dimension is coded. The second processor displays on the display an image showing information related to the dimensions together with an image represented by the image data;
The third processor of the robot control device
Recognizing information regarding the size from a captured image obtained by capturing an image represented by the image data displayed on the display of the second device and an image indicating information regarding the size by the visual sensor;
A teaching system according to claim 5 , wherein information relating to the recognised dimensions is used in carrying out the calibration or position detection. The teaching system of claim 3 or 5, wherein the first processor of the first device generates information about the dimensions based on the image data and information about the resolution and size of the display of the second device. The second processor of the second device
The teaching system according to claim 1 , further comprising: a memory unit for storing information on new image data that can be used for the calibration or position detection and dimensions of the new image data, the memory unit being configured to store the information on the new image data and dimensions of the new image data, based on a user operation. the second device is a teaching pendant connected to a robot control device,
The teaching system of claim 1, wherein the second processor of the second device acquires from the robot control device information regarding the brightness of an image captured by the visual sensor of the second device displaying an image represented by the image data on the display, and adjusts the display brightness of the display that displays the image based on the information regarding the brightness. the storage unit of the first device further stores an application program for displaying an image represented by the image data;
The second processor obtains the application program together with the image data from the first device;
The teaching system according to claim 1 , wherein the second processor executes the application program to display an image represented by the image data on a display of the first device. the application program further has a function of calculating information regarding dimensions when an image represented by the image data is displayed on a display of a device that executes the application, based on information regarding the display of the device;
The second processor,
(1) displaying an image showing the information about the dimensions obtained by executing the application together with an image represented by the image data;
(2) transmitting information regarding the dimensions to the first device;
The teaching system according to claim 13, wherein the teaching system executes any one of the processes above. A teaching device used for teaching regarding calibration or position detection,
A display and
A processor,
The processor,
Acquiring image data for the calibration or position detection from an external device;
A teaching device that displays an image represented by the image data on the display to be captured by a visual sensor in performing the calibration or position detection. The processor further acquires, from the external device, information regarding a size of the image represented by the image data when the image is displayed on the display;
The processor,
(1) displaying, on the display, an image showing information related to the dimensions together with the image represented by the image data;
(2) transmitting information about the dimensions to a robot control device that controls the calibration or position detection;
The teaching device according to claim 15, wherein the teaching device executes any one of the processes above. The processor calculates information about dimensions of the image based on the image data when the image is displayed on the display based on the acquired image data and information about a resolution and a size of the display;
The processor,
(1) displaying, on the display, an image showing information related to the dimensions together with the image represented by the image data;
(2) transmitting information about the calculated dimensions to a robot control device that controls the calibration or position detection;
The teaching device according to claim 15, wherein the teaching device executes any one of the processes above. The processor further comprises:
The teaching device according to claim 15 , further comprising: a memory unit for registering new image data usable for the calibration or position detection and information regarding dimensions of the new image data in the storage unit of the external device based on a user operation. the external device is a robot control device that performs the calibration or position detection,
The teaching device of claim 15, wherein the processor acquires information regarding the brightness of an image captured by the visual sensor of the teaching device displaying an image represented by the image data on the display from the robot control device, and adjusts the display brightness of the display that displays the image based on the information regarding the brightness. A robot control device,
A processor;
A storage unit that stores image data for calibration or position detection,
The processor,
Transmitting the image data to an external device in response to a request from the external device;
A robot control device that performs the calibration or position detection based on a captured image obtained by capturing, by a visual sensor, an image represented by the image data displayed on a display of the external device. the storage unit stores information about a size of the image on a display of the external device when the image represented by the image data is displayed on the display of the external device in association with the image data;
The robot controller of claim 20 , wherein the processor uses information about the dimensions in performing the calibration or the position detection. The processor,
The robot control device according to claim 21 , wherein the dimensional information is calculated based on the image data and information about a resolution and a size of the display of the external device. an image of information regarding a size of the image when the image is displayed on the display of the external device is displayed together with the image represented by the image data;
The processor,
Recognizing information regarding the size from a captured image obtained by capturing an image represented by the image data and an image represented by the information regarding the size displayed on the display of the external device by the visual sensor;
The robot controller according to claim 20 , wherein information regarding the recognized dimensions is used in performing the calibration or position detection. The computer processor
A procedure for storing image data for calibration or position detection in a storage unit;
transmitting the image data to an external device in response to a request from the external device;
A procedure for performing the calibration or position detection based on an image captured by a visual sensor of an image represented by the image data displayed on a display of the external device, and a program for causing the procedure to be executed.

Images