JP7576891B1 - Control device, control method, and program - Google Patents

Abstract

A robot device is enabled to be more appropriately controlled.
[Solution] The control device includes an acquisition unit that acquires measurement information obtained using a sensor for measuring the operating state of the robot device, and a control unit that repeatedly generates control commands for the robot device based on the measurement information, and the control unit derives a speed relationship value for the robot device from the measurement information or the control command, and generates the control command so as to reduce the difference between the speed relationship value and a speed relationship target value that is variable depending on the operating state of the robot device.
[Selected figure] Figure 10

Description

This disclosure relates to a control device, a control method, and a program for controlling a robot device.

Conventionally, control devices for controlling robotic devices have been widely used. In such control devices, a technology is known in which a visual sensor and a force sensor are used to measure the operating state of the robotic device, and the operating state of the robotic device is controlled to be a target state (see, for example, Patent Document 1).

Patent No. 7295344

It is desirable to realize control that does not cause operational errors when using a robotic device, and also to realize control that can handle precise work involving nonlinear operations, for example, to realize highly accurate position control and/or speed control even if errors and/or characteristic changes occur in the sensors, control devices, and/or drive units.

The present disclosure provides a control device, a control method, and a program that enable more appropriate control of a robot device.

The control device according to the first aspect of the present disclosure includes an acquisition unit that acquires measurement information obtained using a sensor for measuring the operating state of a robot device, and a control unit that repeatedly generates control commands for the robot device based on the measurement information, and the control unit derives a speed relationship value for the robot device from the measurement information or the control command, and generates the control command so as to reduce the difference between the speed relationship value and a speed relationship target value that is variable depending on the operating state of the robot device.

A control method according to a second aspect of the present disclosure includes acquiring measurement information obtained using a sensor for measuring an operating state of a robot device, and repeatedly generating a control command for the robot device based on the measurement information, where repeatedly generating the control command includes deriving a speed relationship value for the robot device from the measurement information or the control command, and generating the control command so as to reduce a difference between a speed relationship target value that is variable depending on the operating state of the robot device and the speed relationship value.

A program according to a third aspect of the present disclosure causes a control device to acquire measurement information obtained using a sensor for measuring the operating state of a robot device, and repeatedly generate control commands for the robot device based on the measurement information, where repeatedly generating the control commands includes deriving a speed relationship value for the robot device from the measurement information or the control command, and generating the control command so as to reduce the difference between the speed relationship value and a speed relationship target value that is variable depending on the operating state of the robot device.

According to one aspect of the present disclosure, it is possible to provide a control device, a control method, and a program that enable more appropriate control of a robot device.

FIG. 1 is a diagram illustrating an example of a system configuration of a control system including a control device according to an embodiment. FIG. 2 is a block diagram showing a functional block configuration of a control device according to the first embodiment. FIG. 2 is a diagram for explaining an example of data collection by the control device according to the first embodiment. FIG. 2 is a diagram for explaining an example of data collection by the control device according to the first embodiment. FIG. 4 is a flow chart for explaining an example of data collection by the control device according to the first embodiment. 4 is a diagram for explaining an example of a setting library stored in a control device according to the embodiment; FIG. FIG. 11 is a block diagram showing a functional block configuration of a control device according to a second embodiment. FIG. 11 is a flow chart for explaining an example of a control flow by a control device according to a second embodiment. 13 is a diagram for explaining a specific example of control during the operation of "mounting components on a board" as control by the control device according to the second embodiment. FIG. FIG. 11 is a block diagram showing a functional block configuration of a control device according to a third embodiment. 13 is a diagram for explaining a specific example of control during the operation of "mounting components on a board" as control by the control device according to the third embodiment. FIG.

The embodiments will be described with reference to the drawings. In the description of the drawings, the same or similar parts are denoted by the same or similar reference numerals.

(1) Overview of Embodiments A control device according to an embodiment includes an acquisition unit that acquires measurement information obtained using a sensor for measuring an operating state of a robotic device, and a control unit that repeatedly generates control commands for the robotic device based on the measurement information, wherein the control unit derives a speed relationship value for the robotic device from the measurement information or the control command, and generates the control command so as to reduce a difference between the speed relationship value and a speed relationship target value that is variable depending on the operating state of the robotic device.

In this way, by generating a control command to reduce the difference between the speed-related value and the speed-related target value for the robot device, control that does not cause operational errors during work using the robot device becomes possible. In addition, by making the speed-related target value variable depending on the operating state of the robot device, it becomes possible to handle precise work involving non-linear operations. For example, even if an error and/or characteristic change occurs in the sensor, control device, and/or drive unit, the operation is performed to reduce the difference between the speed-related value and the speed-related target value, making it possible to achieve position control and/or speed control with significantly higher accuracy.

Note that the term "robot device" refers to any device that can operate according to control commands output by a control device, and any robot device is applicable. For example, the robot device may be an industrial robot such as a manipulator, or may include a mobile body that can move automatically. Industrial robots include, for example, vertical articulated robots, SCARA robots, parallel link robots, Cartesian robots, and collaborative robots. Mobile bodies that can move automatically include, for example, drones, vehicles that are configured to be able to drive themselves, unmanned guided vehicles, or mobile robots, as well as combinations with the above industrial robots. In the embodiment described below, an example in which the robot device is a manipulator will be mainly described.

A "control command" is related to the control of the operation of a robotic device, and is, for example, a target control amount, an operation amount, etc. "Outputting a control command" may mean directly controlling a robotic device based on the control command, or, if the robotic device is equipped with a controller, may include outputting a control command to the controller to cause the controller to control the operation of the robotic device.

An "operation" is a job to be performed by a robotic device, and may include multiple processes. Examples of an operation include transporting parts, fitting parts, screwing, processing, etc. An operation may also be a simple job such as gripping a workpiece and releasing the workpiece. An operation may be given in advance, or may be given by specification of an operator.

An "operating state of a robotic device" refers to a state related to the operation of a part of the configuration of the robotic device (e.g., an end effector) and/or a state related to an object of work performed using the robotic device. Furthermore, a "measurement related to the operating state of a robotic device" refers to a measurement of a state related to the operation of a part of the configuration of the robotic device (e.g., an end effector) and/or a measurement of a state related to an object of work performed using the robotic device. An "object" is an object that may be related to the operation of the robotic device, such as a workpiece. A part of the configuration of a robotic device (e.g., an end effector) may also be considered an object.

The "sensor" may be any sensor capable of measuring the operating state of the robot device. In the following description of the embodiment, an example will be described in which the sensor includes at least one of a visual sensor and a force sensor.

The "measurement information" is not limited to the sensor's measurement data itself, but may be feature quantities calculated from the measurement data. For example, the "measurement information" may be information measured or calculated using an encoder and/or a servo motor that are present as basic components of the robot.

In an embodiment, the control unit may repeatedly generate a control command for the robot device for each control period based on the measurement information. Here, the control unit may generate a control command so as to reduce the difference between the speed-related target value and the speed-related value, which is variable for each control period depending on the operating state of the robot device. The "control period" does not necessarily have to be a period of a fixed time length, and may be a period of a variable time length or a period depending on the load (e.g., computational load) of the control device.

(2) First embodiment In the first embodiment, data collection for collecting data to be used for control during actual work before the work will be mainly described. Details of the control during actual work will be described in the second and third embodiments.

(2.1) System Configuration Fig. 1 is a diagram showing an example of the system configuration of a control system including a control device 100 according to the first embodiment. Here, the description will be focused on the hardware configuration of the control system.

In the illustrated example, the robot device 200 is a manipulator. Specifically, the robot device 200 (manipulator) is a six-axis vertical articulated industrial robot, and has a base 221 and six joints 211 to 216. Each of the joints 211 to 216 has a built-in servo motor (not shown) and is configured to be rotatable around each axis.

The first joint 211 is connected to the base 221 and rotates the tip portion around the axis of the base. Instead of the base 221, a moving mechanism capable of automatic movement (self-propelled) may be provided. The second joint 212 is connected to the first joint 211 and rotates the tip portion in the front-rear direction. The third joint 213 is connected to the second joint 212 via a link 222 and rotates the tip portion in the up-down direction. The fourth joint 214 is connected to the third joint 213 via a link 223 and rotates the tip portion around the axis of the link 223. The fifth joint 215 is connected to the fourth joint 214 via a link 224 and rotates the tip portion in the up-down direction. The sixth joint 216 is connected to the fifth joint 215 via a link 225 and rotates the tip portion around the axis of the link 225. A gripper 226 is attached to the tip side of the sixth joint 216 along with a force sensor 320. The gripper 226 is an example of an end effector.

The visual sensor 310 is a sensor that performs image measurement, and is positioned to observe each object (gripper 226, workpiece W1, workpiece W2) that exists in the environment (work space) in which the robot device 200 operates. In the illustrated example, the visual sensor 310 is attached to the link 225, and the visual sensor 310 is provided integrally with the robot device 200. However, the visual sensor 310 may be fixed to equipment in the work space, and the visual sensor 310 may be provided separately from the robot device 200. For example, a camera such as a digital camera or a video camera may be used as the visual sensor 310. The measurement data (i.e., image data) of the visual sensor 310 is an example of visual measurement information.

The force sensor 320 is a sensor that performs force measurement and is configured to measure the force and moment acting on the robot device 200 (specifically, the gripper 226). The force sensor 320 may be, for example, a six-axis force sensor that measures forces acting on the gripper 226 in the three axial directions of the X-axis, Y-axis, and Z-axis, and moments around the X-axis, Y-axis, and Z-axis. That is, the force sensor 320 can measure the force and moment caused by contact between an assembly part supported by the robot device 200 or the gripper 226 and an article. The measurement data of the force sensor 320 may be used to adjust the gripping force of the gripper 226 or to detect whether an abnormal force is acting on the gripper 226. In the following description, the term "force" is used as a term that also includes the meaning of "moment."

The force sensor 320 may be realized, for example, by measuring the current value of a motor (not shown) built into each joint 211 to 216 to measure forces in the three axial directions of the X-axis, Y-axis, and Z-axis, and moments around the X-axis, Y-axis, and Z-axis. Alternatively, it may be a pressure sensor provided on the surface of the robot 200, or a sensor that utilizes changes in the state of a jacket provided on the surface of the robot 200. The sensor may detect changes in the flow rate of air and/or liquid, changes in capacitance, etc.

Each of the joints 211 to 216 may have an encoder (not shown). The encoder is an example of a sensor. The encoder is configured to be able to measure the angle (control amount) of each of the joints 211 to 216. The measurement data of the encoder may be used to control the angle of each of the joints 211 to 216.

The control system may have a transport device 510 that transports the workpiece W2. The robot device 200 can perform work using a gripper 226 (end effector) attached to the tip of the arm. The end effector is an external device that can be replaced depending on the application, and a welding gun, tool, etc. may be attached instead of the gripper 226. The robot device 200 can perform work using a force sensor 320 while tracking the workpiece W2 moving on the transport device 510 using a visual sensor 310. In the illustrated example, the robot device 200 performs work by fitting the workpiece W1, which is an assembly part grasped by the robot device 200, into a hole in the workpiece W2 (part to be assembled, for example, a circuit board) moving on the transport device 510.

The control device 100 has a processor 101, a memory 102, and an external interface (I/F) 103. The processor 101 includes a CPU (Central Processing Unit). The processor 101 may further include at least one of a microprocessor, an FPGA (field-programmable gate array), and a DSP (digital signal processor). The memory 102 includes a RAM (Random Access Memory), a ROM (Read Only Memory), and an auxiliary storage device (e.g., a hard disk drive, a solid state drive). The processor 101 and the memory 102 constitute a computer. The control device 100 may be composed of multiple computers. The control device 100 is not limited to an information processing device designed specifically for the service provided, but may be a general-purpose information processing device such as a personal computer (PC) or a controller such as a programmable logic controller (PLC).

The memory 102 stores a program executed by the processor 101. The processor 101, together with the memory 102, executes the program stored in the memory 102 to realize the functions of each functional block described below. Details will be described later, but the memory 102 may store, for example, a recognition library including a trained model used for image recognition to recognize an object, and a setting library including data collected by data collection before a task and setting information acquired based on the collected data. The recognition library and the setting library may be provided for each type of task that the robot device 200 can perform. The setting information included in the setting library may include a trained model generated based on the collected data.

The external I/F 103 is, for example, a USB (Universal Serial Bus) port or a dedicated port, and is an interface for connecting to an external device so as to be able to communicate with the external device. The external I/F 103 may be connected to an external device (including the robot device 200) by wire or wirelessly. The type and number of the external I/F 103 may be appropriately selected according to the type and number of the external devices to be connected. In the illustrated example, the control device 100 is connected to the robot device 200, the visual sensor 310, and a user interface (I/F) 400 via the external I/F 103. As shown in FIG. 2, the user I/F 400 includes a display device 410 and an operation device 420. In the illustrated example, the user I/F 400 is provided separately from the control device 100, but the user I/F 400 may be provided integrally with the control device 100. The display device 410 may be a liquid crystal display or an organic EL (Electro-Luminescence) display. The display device 410 may be a display equipped with a speaker. The operation device 420 is, for example, a device for performing operation input such as a keyboard, a mouse, or a touch panel. The display device 410 and the operation device 420 may be integrated as a touch panel display. An operator can check the status of the control device 100 and operate the control device 100 by using the display device 410 and the operation device 420.

(2.2) Functional Block Configuration of the Control Device FIG. 2 is a block diagram showing a functional block configuration of the control device 100 according to the first embodiment. In the first embodiment, a functional block configuration related to data collection that collects data to be used for control during work before the work is performed will be mainly described. Such data collection may be performed before a user (including an operator) starts using the control device 100 (and the robot device 200). For example, data collection may be performed in advance before the shipment of the control device 100 (and the robot device 200).

The control device 100 has an action generating unit 110, an acquiring unit 120, a data collecting unit 130, a data storing unit 140, a setting acquiring unit 150, and a library storing unit 160. In this embodiment, the control device 100 may or may not have a control unit 170.

The motion generating unit 110 generates a control command for executing a forward motion that changes the motion state of the robot device 200 from an arbitrary state to a target state. The motion generating unit 110 also generates a control command for executing a reverse motion that changes the motion state of the robot device 200 from a target state to an arbitrary state different from the target state. The motion generating unit 110 outputs the generated control command to the driving unit 210 of the robot device 200. The driving unit 210 includes servo motors provided at the joints 211 to 216 of the robot device 200. The driving unit 210 may include a controller on the robot device 200 side. The driving unit 210 drives the servo motors according to the control command to operate the robot device 200.

The goal state here refers to a state that is realized when the purpose of a task (or process) is achieved, or an intermediate state of the task. In the first embodiment, a series of tasks is assumed in which the robot device 200 attaches an assembly part (workpiece W1) carried by an end effector (gripper 226) to a part to be assembled (workpiece W2), so the goal state is a state in which the assembly part is attached to the part to be assembled, or an intermediate state before attachment. Specifically, the goal state is a state in which the workpiece W1 is fitted into a hole in the workpiece W2, or an intermediate state in which the workpiece W1 is in contact with the workpiece W2 at a position where it can be fitted into the hole.

The arbitrary state is a state different from the target state, for example, a state in which the workpiece W1 is located away from the hole in the workpiece W2. The arbitrary state may be set by operation input (user input) via the operation device 420. For example, the position of the arbitrary state may be set by operation input (user input) based on the position of the target state. Note that the term "position" may include not only the meaning of "coordinates" but also the meaning of "posture".

The acquisition unit 120 acquires measurement information obtained using a sensor 300 for measuring the operating state of the robot device 200. In the illustrated example, the sensor 300 includes a visual sensor 310 and a force sensor 320. The sensor 300 may further include other sensors 330 such as an encoder. The acquisition unit 120 may include an image recognition unit 121 that performs image recognition on the measurement data (i.e., image data) output by the visual sensor 310. The image recognition unit 121 may recognize an object (e.g., the gripper 226, the workpiece W1, the workpiece W2) by image recognition such as feature extraction, and acquire the position of the object (e.g., the coordinates of the feature). Such position information is an example of visual measurement information.

The data collection unit 130 repeatedly collects data including a set of the control command output by the motion generation unit 110 and the measurement information acquired by the acquisition unit 120 during the execution of the reverse motion according to the control command output by the motion generation unit 110. This makes it possible to efficiently collect data that can be used to control the motion (forward motion) of the robot device 200 during actual work. For example, during actual work using the robot device 200, it becomes possible to appropriately control the motion of the robot device from an arbitrary state to a target state by generating a control command from the measurement information obtained during work based on the collected data. Note that the data collection unit 130 may repeatedly collect data including a set of the control command and the measurement information acquired by the acquisition unit 120 during the execution of the forward motion according to the control command output by the motion generation unit 110.

The motion generation unit 110 may generate control commands for executing multiple patterns of reverse motion that change the motion state of the robot device 200 from a target state to multiple arbitrary states that are different from each other. The data collection unit 130 may repeatedly collect data for each of the multiple patterns of reverse motion. This allows multiple patterns of data corresponding to multiple movement paths to be collected, making it possible to provide versatility to the motion control of the robot device 200 during actual work. For example, even if the workpiece W2 is not fixed or moves during work, it becomes easy to attach the workpiece W1 to the workpiece W2.

The motion generation unit 110 may generate control commands that collect more data in areas closer to the target state, and generate control commands that collect less data in areas farther from the target state. The closer an area is to the target state, the more precise and accurate control is required during actual work. By increasing the amount of data collected in areas closer to the target state, it becomes possible to collect a sufficient amount of data to perform precise and accurate control. On the other hand, in areas farther from the target state, precise and accurate control is not as necessary. Therefore, by generating control commands that collect less data in areas farther from the target state, unnecessary data collection is suppressed, enabling efficient data collection.

The acquisition unit 120 may shorten the acquisition cycle (e.g., sampling frequency) so that the amount of measurement information acquired increases in areas closer to the target state, and may extend the acquisition cycle so that the amount of measurement information acquired increases in areas farther from the target state. This type of processing also makes it possible to collect more data in areas closer to the target state and less data in areas farther from the target state.

The acquisition unit 120 may acquire the difference between the target measurement information obtained in the target state and the current measurement information obtained during the execution of the reverse direction operation as the relative measurement information. The data collection unit 130 may repeatedly collect data including a set of the control command and the relative measurement information during the execution of the reverse direction operation according to the control command. For example, the acquisition unit 120 may acquire the difference between the visual measurement information (target position) of the object obtained by the visual sensor 310 in the target state and the current measurement information (current position) of the object obtained during the execution of the reverse direction operation as the relative position information. In the environment of FIG. 1, the acquisition unit 120 may acquire the difference between the target position and each current position as the relative position by setting the position of the hole in the work W2 as the target position and setting each position on the movement path of the work W1 as the current position. As a result, a control command is associated with each relative positional relationship between the target position and the current position of the object. Therefore, during actual work, it becomes possible to generate an appropriate control command from the relative positional relationship between the target position and the current position of the object. Furthermore, control using relative positional relationships can be applied even when the target object moves during work.

The acquisition unit 120 may acquire measurement information including a speed-related value related to the robot device 200 in response to the output of the sensor 300 or a control command. The speed-related value related to the robot device 200 may be at least one of the speed, acceleration, and jerk of the object. The acquisition unit 120 may derive the speed, acceleration, and jerk of the object from visual measurement information (position information of the object) obtained by the visual sensor 310. The acquisition unit 120 may derive the speed, acceleration, and jerk of the object from a control command output to the drive unit 210. The acquisition unit 120 may derive the speed, acceleration, and jerk of the object from measurement information obtained by another sensor 330 (e.g., an encoder). The data collection unit 130 may repeatedly collect data including a set of the control command and measurement information including the speed-related value during the execution of a reverse movement according to the control command. The speed-related value collected in this manner can be used when setting a speed-related target value in control during actual work. Control using the speed-related target value will be described in the third embodiment.

The data storage unit 140 stores the data collected by the data collection unit 130. The data collected by the data collection unit 130 includes multiple sets of measurement information (relative measurement information) and control commands. Each set may include measurement information (relative measurement information), a speed relationship value, and a control command. The data collected by the data collection unit 130 may include more sets of data for areas closer to the target state.

The setting acquisition unit 150 acquires setting information for performing a task using the robot device 200 based on data collected by the data collection unit 130 (specifically, data stored in the data storage unit 140). Setting information for performing a certain task is called the "setting library" for that task. For example, the setting acquisition unit 150 acquires a setting library after data collection by the data collection unit 130 is completed. The library storage unit 160 stores the setting library acquired by the setting acquisition unit 150. The control unit 170 uses the setting library to control the robot device 200 during actual work.

The setting library includes operation parameters for each work process, and information on the transition destination and transition conditions for each process (such as normal values, timeout values, and error values for the transition conditions). The operation parameters may include correspondence information that corresponds measurement information (relative measurement information) to a control command, or may include correspondence information that corresponds measurement information (relative measurement information), a speed relationship value, and a control command. The operation parameters may include a control ratio between visual control, which is robot control based on visual measurement information obtained by the visual sensor 310, and force control, which is robot control based on force measurement information obtained by the force sensor 320. The operation parameters may include information on the type of object (metal, resin, screw, connector, etc.). The operation parameters may include information on features on the object (holes, edge surfaces, connectors, etc.). The setting acquisition unit 150 may acquire the setting library at least partially based on the operation input performed via the operation device 420.

The setting acquisition unit 150 may use the data collected by the data collection unit 130 as learning data to acquire a learned model for deriving a control command from measurement information by machine learning. In this case, the control unit 170 uses a setting library including the learned model during actual work to control the robot device 200 based on measurement information obtained during work. The type of the learned model (learning model) is not particularly limited as long as the ability to make inferences for generating control commands can be acquired by machine learning. The type of machine learning is not particularly limited, but is typically supervised learning or reinforcement learning. The learning model may be configured, for example, by a neural network such as a deep neural network (DNN). The learning model may be configured, for example, by a value function such as a state value function or an action value function. In this way, the setting acquisition unit 150 determines a calculation for inferring an optimal value, enables inference, and generates an optimal value variable for a control command by machine learning.

The setting acquisition unit 150 may acquire a setting library that includes the control ratio between visual control and haptic control, for example, for each process. In this case, the control unit 170 uses the setting library during actual work to dynamically or gradually change the control ratio according to the status of the work. Details of such control will be described in the second embodiment.

The setting acquisition unit 150 may acquire a setting library including speed-related values related to the robot device 200, for example, for each process. In this case, the control unit 170 may derive the speed-related values related to the robot device 200 from measurement information or control commands obtained during work. The control unit 170 may then repeatedly control the robot device 200 to reduce the difference between the speed-related values and the speed-related target values, based on the setting library and the measurement information obtained during work. Details of such control will be described in the third embodiment.

The motion generation unit 110 may generate a control command for executing a reverse motion for each type of task performed using the robot device 200. In the environment of FIG. 1, the task type is "mounting components on a circuit board." Other examples of task types include "packing food into boxes," "screw tightening," "pick and place," and "AGV (Automated Guided Vehicle) control." The data collection unit 130 may collect data for each task type by repeatedly collecting data for each task type.

In this case, the setting acquisition unit 150 can acquire a setting library for each type of work based on the data collected by the data collection unit 130 for each type of work. The control unit 170 then uses the setting library corresponding to the type of work to be actually performed to control the robot device 200 to perform that work. For example, the control unit 170 may display a list of setting libraries stored in the library storage unit 160 on the display device 410, and control the robot device 200 using a setting library selected from the list using the operation device 420. Note that if the library storage unit 160 also stores a recognition library in association with the setting library, image recognition for performing the work may be performed using the recognition library corresponding to the type of work to be actually performed.

(2.3) Example of Data Collection FIGS. 3 and 4 are diagrams for explaining an example of data collection by the control device 100 according to the first embodiment.

As described above, the motion generation unit 110 generates control commands for executing a plurality of patterns of reverse motions that change the motion state of the robot device 200 from a target state to a plurality of arbitrary states different from each other. The data collection unit 130 repeatedly collects data for each of the plurality of patterns of reverse motions. As shown in FIG. 3, the motion generation unit 110 generates control commands that collect more data in an area closer to the target state, and generates control commands that collect less data in an area farther from the target state. In FIG. 3, the x-axis and y-axis indicate directions that are mutually orthogonal in a horizontal plane, and the z-axis indicates a vertical direction. Also, in FIG. 3, the amount of collected data is indicated by shading. Specifically, the more the amount of collected data, the darker the color, and the less the amount of collected data, the lighter the color.

In the example of FIG. 4, the motion generation unit 110 causes the robot device 200 to execute reverse motion to any of eight patterns P1 to P8, with the state in which the workpiece W1 (component) is in the hole of the workpiece W2 (circuit board) as the target state. Although the movement path of each pattern P1 to P8 is illustrated as being linear, each movement path does not have to be linear. For example, the movement path may be such that the workpiece W1 (component) is moved horizontally or diagonally upward after being pulled out upward from the hole of the workpiece W2 (circuit board). In addition, among the patterns P1 to P8, the movement of P2, P4, and P8 ends in the region R close to the target state. As a result, the amount of data collected increases the closer the point is to the position of the target state.

(2.4) Specific Example of Operation of the Control Device FIG. 5 is a flow diagram for explaining an example of data collection by the control device 100 according to the first embodiment.

In step S101, a user (e.g., a worker) moves an object (e.g., a workpiece W1 grasped or supported by an end effector) to a target state. The motion generation unit 110 also acquires sequence information of a control command corresponding to any one of a plurality of reverse motion patterns (i.e., any arbitrary state) and/or information on the position of the arbitrary state. Here, the acquisition unit 120 acquires the position of the object in the target state as a target position from visual measurement information obtained using the visual sensor 310. The acquisition unit 120 acquires the force (reaction force) applied to the object in the target state as a target force (target reaction force) from force measurement information obtained using the force sensor 320.

In step S102, the motion generation unit 110 outputs a control command to the robot device 200 (drive unit 210) based on the information acquired in step S101, thereby moving the target object a predetermined distance.

In step S103, the acquisition unit 120 acquires current measurement information. The acquisition unit 120 acquires the current position of the object from the visual measurement information obtained using the visual sensor 310. The acquisition unit 120 may acquire the difference between the current position of the object and the target position as a relative position (amount of position change). The acquisition unit 120 also acquires the current force (reaction force) from the force measurement information obtained using the force sensor 320. The acquisition unit 120 may acquire the difference between the current force (reaction force) and the target force (reaction force) as a relative force (amount of force change). Furthermore, the acquisition unit 120 may acquire the speed-related values (speed, acceleration, and jerk) of the object from the current visual measurement information or the control command of step S102.

In step S104, the data collection unit 130 collects a set of the control command in step S102 and the information acquired in step S103 (position change amount, force change amount, and velocity relationship value), and stores the set in the data storage unit 140.

If the movement of the object to the arbitrary state corresponding to the current reverse motion pattern is not complete (step S105: NO), the process returns to step S102, and the motion generation unit 110 outputs a control command to the robot device 200 (drive unit 210) to move the object by a predetermined amount. Then, in step S103, the acquisition unit 120 acquires information (position change amount, force change amount, speed-related value). In step S104, the data collection unit 130 collects a set of the control command of step S102 and the information acquired in step S103 (position change amount, force change amount, speed-related value), and stores them in the data storage unit 140. This process is repeated until the movement to the arbitrary state corresponding to the current reverse motion pattern is complete.

On the other hand, if the movement of the object to an arbitrary state corresponding to the current reverse motion pattern is complete (step S105: YES), the motion generation unit 110 checks whether the motions have been completed for all of the multiple reverse motion patterns. If the motions have not been completed for all patterns (step S106: NO), the process moves to the next reverse motion pattern (step S107) and resumes from step S101.

If all patterns have been completed (step S106: YES), in step S108, the setting acquisition unit 150 determines the operating conditions (operation parameters) for performing work using the robot device 200 based on the data collected by the data collection unit 130 (specifically, the data stored in the data storage unit 140). Then, in step S109, the setting acquisition unit 150 stores the setting information including the operating conditions (operation parameters) determined in step S108 in the library storage unit 160 as a setting library.

Note that the flow shown in FIG. 5 may be executed for each type of work, so that a setting library for each type of work may be stored in the library storage unit 160.

(2.5) Example of Setting Library FIG. 6 is a diagram for explaining an example of a setting library stored in the control device 100 according to the embodiment.

In the illustrated example, the types of work include "mounting components on a circuit board," "packing food into boxes," "tightening screws," "pick and place" (also known as "picking"), and "AGV control." Each work involves multiple steps.

For example, in a task type called "mounting components on a board," the steps are performed in the following order: step 1 "recognize component → move" → step 2 "grasp component" → step 3 "move component toward board" → step 4 "approach" → step 5 "recognize holes in board" → step 6 "approach hole" → step 7 "hole tracing operation" → step 8 "insert hole" → step 9 "hole insertion completed" → step 10 "release grip." On the other hand, when collecting data, the control device 100 may control the operation of the robot device 200 in the reverse order to this order. In the case of "AGV control," a step of avoiding obstacles may be further included. The step of avoiding obstacles may be included in step 4 "move object toward target object."

The setting library includes operation parameters for each work process, and information on the transition destination and transition conditions for each process (normal value, timeout value, error value, etc. of the transition condition). The operation parameters may include correspondence information that corresponds measurement information (relative measurement information) with a control command, or may include correspondence information that corresponds measurement information (relative measurement information), a speed relationship value, and a control command. The operation parameters may include a control ratio between visual control and force control. The operation parameters may include information on the type of object (metal, resin, screw, connector, etc.). The operation parameters may include information on features on the object (holes, edges, connectors, etc.).

(3) Second embodiment The second embodiment will be described mainly with respect to differences from the first embodiment. The system configuration according to the second embodiment is similar to that of the first embodiment (see FIG. 1). Note that the second embodiment will be described mainly as an example in which the second embodiment is based on the first embodiment, but the second embodiment does not necessarily have to be based on at least a part of the first embodiment.

(3.1) Functional Block Configuration of the Control Device FIG. 7 is a block diagram showing the functional block configuration of the control device 100 according to the second embodiment.

The control device 100 according to the second embodiment has an acquisition unit 120 that acquires measurement information obtained using a sensor 300 for measuring the operating state of the robot device 200, and a control unit 170 that controls the operation of the robot device 200 based on the measurement information. The sensor 300 includes a visual sensor 310 and a force sensor 320. The control unit 170 dynamically or stepwise changes the control ratio between visual control, which is control based on visual measurement information obtained by the visual sensor 310, and force control, which is control based on force measurement information obtained by the force sensor 320, depending on the situation of the work using the robot device 200. This makes it possible to utilize the advantages of visual control and force control, and appropriately control the operation of the robot device 200 by achieving both visual control and force control.

In the second embodiment, the control unit 170 has a visual control unit 171A that refers to the visual measurement information and generates first information indicating the content of control of the robot device 200, a force sense control unit 171B that refers to the force sense measurement information and generates second information indicating the content of control of the robot device 200, a command generation unit 172 that generates a control command for the robot device 200 based on the first information and the second information, and a weighting unit 173 that changes the control ratio by performing weighting processing between the visual measurement information and the force sense measurement information, or between the first information and the second information, depending on the situation of the work using the robot device 200. This makes it possible to appropriately change the control ratio by the weighting processing.

The visual control performed by the visual control unit 171A may be performed, for example, based on the operation parameters (such as correspondence information) in the setting library. The visual control unit 171A identifies the relative positional relationship between the current position and the target position of the object based on the visual measurement information obtained using the visual sensor 310. The visual control unit 171A may then generate a visual control command from the identified positional relationship using the correspondence information to reduce the difference between the current position and the target position (i.e., move the object closer to the target position), and output this visual control command as the first information. At least a part of such visual control may be performed using a trained model.

Similarly, the force-sense control performed by the force-sense control unit 171B may be performed based on, for example, operation parameters (such as correspondence information) in the setting library. The force-sense control unit 171B may identify the difference between the current force (current reaction force) of the object and the target force (target reaction force) based on the force-sense measurement information obtained using the force sensor 320, generate a force-sense control command from the identified difference to reduce the difference using the correspondence information, and output this force-sense control command as the second information. At least a part of such force-sense control may be performed using a trained model.

The control unit 170 (weighting unit 173) may dynamically or gradually change the control ratio as the work using the robotic device 200 progresses. As described above, the work using the robotic device 200 may include a number of predetermined steps. The control unit 170 may change the control ratio for each step. This makes it possible to control the operation of the robotic device 200 with an appropriate control ratio for each step.

The library storage unit 160 may store a setting library (setting information) including settings related to the control ratios of each of a plurality of processes. The control unit 170 (weighting unit 173) may change the control ratio for each process based on the setting library. This allows an appropriate control ratio to be set for each process.

When transitioning from one process to the next process in a task, the control unit 170 (weighting unit 173) gradually changes the control ratio from the control ratio in that process to the control ratio in the next process. This prevents abrupt fluctuations in the control ratio, enabling precise operation control and suppressing the occurrence of operation errors.

For example, if the control ratio (visual control: force feedback control) of process A is set to "80:20" and the control ratio of process B, which follows process A, is set to "50:50," when the transition condition from process A to process B is satisfied, the control unit 170 (weighting unit 173) changes the control ratio stepwise, such as "80:20" → "75:25" → "70:30" → "65:35" → "60:40" → "55:45" → "50:50." Alternatively, the control unit 170 (weighting unit 173) may change the control ratio continuously, such as "80:20" → "79:21" → "78:22" → "77:22" → ... Such a change in the control ratio may be performed in control cycle units, or may be performed in time units consisting of multiple control cycles.

The control unit 170 (weighting unit 173) may determine whether a transition condition from one process to the next process in a task is satisfied based on the measurement information obtained using the sensor 300. The transition condition may be included as one of the operation parameters in the setting library. In response to the transition condition being satisfied, the control unit 170 (weighting unit 173) may transition from the one process to the next process and change the control ratio to the control ratio corresponding to the next process.

If the control in the next step does not converge after switching to the next step, the control unit 170 may return to the one step and change the control ratio to the one step. In this way, if the control does not converge, it is possible to return to the previous step and restart the operation, so that the control can be expected to converge when transitioning to the next step. Note that "if the control does not converge" may mean that at least one of the following conditions is met: the measurement information does not satisfy the normal value of the transition condition, a timeout has occurred, or the measurement information has become an error value.

The control unit 170 (weighting unit 173) may switch the control of the robot device 200 between a first control state (also referred to as "visual-based") in which visual control is given priority, a second control state (also referred to as "visual + force-based") in which visual control and force-sense control are used in cooperation, and a third control state (also referred to as "force-sense-based") in which force-sense control is given priority. "Control with visual control given priority" may mean, for example, that the control ratio accounted for by visual control is approximately 65% to 100%. "Control with visual control and force-sense control used in cooperation" may mean, for example, that the visual control:force-sense control ratio is approximately 50:50. "Control with force-sense control given priority" may mean, for example, that the control ratio accounted for by force-sense control is approximately 65% to 100%. The control unit 170 may determine switching of control in the first control state to the third control state based on the measurement information obtained using the sensor 300. When the difference between the measurement information and the target value of the measurement information does not become equal to or less than a predetermined value in the first control state (which may be the case when the transition condition is not satisfied), the control unit 170 (weighting unit 173) may switch to the second control state or the third control state and refer to the force measurement information.

When an instruction is given from outside, the control unit 170 (weighting unit 173) may change the control ratio in response to the instruction. For example, when an instruction to change the control ratio is given via the operation device 420, the control unit 170 (weighting unit 173) may change the control ratio in response to the instruction.

The control unit 170 may further include a prediction unit 174A that predicts the control result of the subsequent control content for the robot device 200 according to the control content performed on the robot device 200 and the control result (sensor information) for the control content, and a correction unit 174B that corrects the subsequent control content according to the prediction. For example, when the robot device 200 does not perform the operation specified in the control command due to an external factor such as an error factor on the robot device 200 side or an error factor on the sensor 300 side, the prediction unit 174A identifies the error (i.e., the difference between the content of the control command and the control result) and predicts the control result for the subsequent control content. Then, the correction unit 174B may correct the control command generated by the command generation unit 172 according to the prediction result of the prediction unit 174A and output it to the drive unit 210. For example, the correction unit 174B may correct the control command so as to cancel the identified error. This makes it possible to perform more accurate robot control even when there is an external error factor.

The library storage unit 160 may store a plurality of setting libraries prepared for each type of work. Each of the plurality of setting libraries may include setting information (operation parameters) of the control ratio. Each of the plurality of setting libraries includes setting information related to each of a series of processes. The control unit 170 controls the robot device 200 using a setting library selected from the plurality of setting libraries according to the type of work to be actually performed. The selection may be performed by an operation input via the operation device 420. Prior to such an operation input, the acquisition unit 120 (image recognition unit 121) may estimate the work to be actually performed based on visual measurement information obtained by the visual sensor 310 observing the work site or the object. The control unit 170 may suggest the setting library corresponding to the estimated work to the user (operator) by displaying it on the display device 410.

The library storage unit 160 may further store a plurality of recognition libraries prepared for each type of work. Each of the plurality of recognition libraries may include a trained model used for image recognition processing of an object. The control unit 170 performs image recognition processing using a recognition library selected from the plurality of recognition libraries according to the type of work to be actually performed. The selection may be performed by an operation input via the operation device 420. Prior to such operation input, the acquisition unit 120 (image recognition unit 121) may estimate the work to be actually performed based on visual measurement information obtained by the visual sensor 310 observing the work site or the object. The control unit 170 may suggest the recognition library corresponding to the estimated work to the user (operator) by displaying it on the display device 410.

(3.2) Operation Flow of the Control Device FIG. 8 is a flow diagram for explaining an example of a control flow by the control device 100 according to the second embodiment.

In step S201, the control unit 170 starts process n. At the start of the work, the value of n is "1". At that time, the control unit 170 applies the setting information (operation parameters including the control ratio) corresponding to process n.

In step S202, the control unit 170 applies the setting information (operation parameters including the control ratio) corresponding to process n to control the operation of the robot device 200.

In step S203, the control unit 170 determines whether the transition condition from process n to the next process is satisfied (i.e., whether process n is completed). If process n is completed (step S203: YES), the process proceeds to step S204. On the other hand, if process n is not completed (step S203: NO), in step S205, the control unit 170 determines whether the control of process n has converged. If it is determined that the control of process n has converged (step S205: YES), the process returns to step S202.

When it is determined that the control of process n has not converged (step S205: NO), the control unit 170 returns to the process (n-1) prior to process n. At that time, the control unit 170 may gradually change the control ratio from the control ratio of process n to the control ratio of process (n-1). Alternatively, instead of returning to process (n-1), the control unit 170 may change at least a part of the operating parameters of process n in accordance with a predetermined rule and then resume process n. For example, if it is estimated that the speed-related target value is too high, the speed-related target value may be lowered by one step and process n resumed.

In step S204, the control unit 170 determines whether or not all steps of the work have been completed. If all steps have been completed, this flow ends. If all steps have not been completed (step S204: NO), in step S207, the control unit 170 proceeds to the next step (n+1) after step n. At that time, the control unit 170 may gradually change the control ratio from the control ratio of step n to the control ratio of step (n+1).

(3.3) Specific Example of Operation of the Control Device FIG. 9 is a diagram for explaining a specific example of control performed by the control device 100 according to the second embodiment during the operation of "mounting components on a board."

In the task type "mounting components on board," the steps are performed in the following order: Step 1 "Recognize and move components" → Step 2 "Grasp components" → Step 3 "Move components toward board" → Step 4 "Approach" → Step 5 "Recognize holes on board" → Step 6 "Approach holes" → Step 7 "Pass holes" → Step 8 "Insert holes" → Step 9 "Hole insertion completed" → Step 10 "Release gripping." The operation parameters for each of these steps are included in the settings library.

Of these processes, basically, force-based control is set for processes involving contact between objects (end effector (gripper 226), component (workpiece W1), board (workpiece W2)), visual-based control is set for processes in which objects (end effector, component) are moved at a certain speed or faster, and visual + force-based control is set for processes in which objects are moved at a low speed toward contact between the objects. However, it is preferable that the specific control ratio for each process is set based on data collected in the data collection according to the first embodiment.

In the illustrated example, vision-based control is applied to step 1, "Part recognition → movement." However, when the position of the part recognized by image recognition is set as the target position and the current position of the end effector approaches the vicinity of the target position, vision + force-based control may be applied. Force-based control is applied to step 2, "Part grip."

Vision-based control is applied to step 3, "Moving the component towards the board." Vision-based control is applied to step 4, "Approach." However, when the current position of the component approaches the vicinity of the target position, which is the position of the board recognized by image recognition, vision- and force-based control may be applied. Vision-based control is applied to step 5, "Recognizing holes in the board."

Vision and force-based control is applied to step 6 "Hole approach". Force-based control is applied from step 7 "Hole tracing operation" to step 10 "Grip release".

(4) Third embodiment The third embodiment will be described mainly with respect to differences from the first and second embodiments. The system configuration according to the third embodiment is similar to that of the first embodiment (see FIG. 1). Note that the third embodiment will be described mainly as an example in which the third embodiment is based on the first and second embodiments, but the third embodiment does not necessarily have to be based on at least a part of the first and second embodiments.

(4.1) Functional Block Configuration of the Control Device FIG. 10 is a block diagram showing a functional block configuration of the control device 100 according to the third embodiment.

The control device 100 according to the third embodiment includes an acquisition unit 120 that acquires measurement information obtained using a sensor 300 for measuring the operating state of the robot device 200, and a control unit 170 that repeatedly generates a control command for the robot device 200 for each control period based on the measurement information. The control unit 170 derives a speed relationship value for the robot device 200 from the measurement information or the control command, and generates a control command to reduce the difference between the speed relationship value and a speed relationship target value that is variable for each control period according to the operating state of the robot device 200. This enables control that does not cause operation errors when working with the robot device 200. In addition, by making the speed relationship target value variable for each control period according to the operating state (work situation) of the robot device 200, it becomes possible to handle precise work involving nonlinear operations. As described above, the speed relationship value includes at least one of speed, acceleration, and jerk.

In the third embodiment, the control unit 170 has a derivation unit 175 that derives a speed-related value from measurement information or a control command for each control cycle, a target setting unit 176 that variably sets a speed-related target value for each control cycle according to the operating state (work situation) of the robot device 200, and a command generation unit 172 that generates a control command for each control cycle so as to reduce the difference between the speed-related value and the speed-related target value. The target setting unit 176 may variably set the speed-related target value according to the speed-related value included in the operating parameters in the setting library.

The command generation unit 172 may incorporate the functions of the visual control unit 171A, the force control unit 171B, and the weighting unit 173 described in the second embodiment. Alternatively, the control unit 170 may be provided with the visual control unit 171A, the force control unit 171B, and the weighting unit 173 described in the second embodiment, in addition to the command generation unit 172.

The control unit 170 (command generation unit 172) generates a control command to reduce the difference between the measurement information and the target measurement information (target value of the measurement information) and to reduce the difference between the speed-related value and the speed-related target value.

For example, the control unit 170 (command generation unit 172) may generate a control command to reduce the difference between the visual measurement information (current position) obtained using the visual sensor 310 and the target visual measurement information (target position), and to reduce the difference between the speed-related value and the speed-related target value. Here, the control to reduce the difference between the visual measurement information (current position) and the target visual measurement information (target position) is similar to the visual control described in the second embodiment.

The control unit 170 (command generation unit 172) may generate a control command to reduce the difference between the force sense measurement information (current force (reaction force)) obtained using the force sensor 320 and the target force sense measurement information (target force (reaction force)), and to reduce the difference between the speed relationship value and the speed relationship target value. Here, the control to reduce the difference between the force sense measurement information (current force (reaction force)) and the target force sense measurement information (target force (reaction force)) is the same as the force sense control described in the second embodiment.

The control unit 170 (goal setting unit 176) may change the speed-related target value according to the difference between the current measurement information and the target measurement information (current relative measurement information). For example, the control unit 170 (goal setting unit 176) may set the speed-related target value to the speed-related value corresponding to the current relative measurement information, using association information included in the setting library that associates measurement information (relative measurement information) with a speed-related value.

For example, the control unit 170 (goal setting unit 176) may change the speed-related target value in accordance with the difference between the current visual measurement information obtained using the visual sensor 310 and the target visual measurement information. The control unit 170 (goal setting unit 176) may change the speed-related target value in accordance with the difference between the current force measurement information obtained using the force sensor 320 and the target force measurement information.

The control unit 170 (derivation unit 175 and goal setting unit 176) may select one or more values from speed, acceleration, and jerk to be used as the speed-related value (and speed-related target value) depending on the difference between the measurement information and the target measurement information (current relative measurement information). For example, the control unit 170 (derivation unit 175 and goal setting unit 176) may select speed as the speed-related value (and speed-related target value) during visual-based control. On the other hand, during force-based control, it may select jerk as the speed-related value (and speed-related target value). During visual + force-based control, it may select acceleration as the speed-related value (and speed-related target value).

As in the second embodiment, the control unit 170 may have a prediction unit 174A that predicts the control result of a subsequent control operation performed on the robot device 200 based on the control operation performed on the robot device 200 and the control result of the control operation, and a correction unit 174B that corrects the subsequent control operation based on the prediction.

The library storage unit 160 may store multiple setting libraries prepared for each type of work. Each of the multiple setting libraries may include setting information for speed-related values (speed-related target values). The control unit 170 controls the robot device 200 using a setting library selected from the multiple setting libraries according to the type of work to be actually performed. The selection may be made by an operation input via the operation device 420. Prior to such operation input, the acquisition unit 120 (image recognition unit 121) may estimate the work to be actually performed based on visual measurement information obtained by the visual sensor 310 observing the work site or the object. The control unit 170 may suggest the setting library corresponding to the estimated work to the user (operator) by displaying it on the display device 410.

The library storage unit 160 may further store a plurality of recognition libraries prepared for each type of work. Each of the plurality of recognition libraries may include a trained model used for image recognition processing of an object. The control unit 170 performs image recognition processing using a recognition library selected from the plurality of recognition libraries according to the type of work to be actually performed. The selection may be performed by an operation input via the operation device 420. Prior to such operation input, the acquisition unit 120 (image recognition unit 121) may estimate the work to be actually performed based on visual measurement information obtained by the visual sensor 310 observing the work site or the object. The control unit 170 may suggest the recognition library corresponding to the estimated work to the user (operator) by displaying it on the display device 410.

(4.2) Specific Example of Operation of the Control Device FIG. 11 is a diagram for explaining a specific example of control performed by the control device 100 according to the third embodiment during the operation of "mounting components on a board."

As mentioned above, in the task type "mounting components on board", the steps are performed in the following order: Step 1 "Recognize and move components" → Step 2 "Grasp components" → Step 3 "Move components towards board" → Step 4 "Approach" → Step 5 "Recognize holes on board" → Step 6 "Approach holes" → Step 7 "Pass holes" → Step 8 "Insert holes" → Step 9 "Hole insertion completed" → Step 10 "Release gripping". The operation parameters for each of these steps are included in the settings library.

In each step, the control unit 170 (command generation unit 172) generates a control command to reduce the difference between the measurement information obtained using the sensor 300 and the target measurement information (target value of the measurement information), and to reduce the difference between the speed-related value and the speed-related target value.

In the illustrated example, in step 1 "Part recognition → movement", "part position" can be applied as the target measurement information, and "speed" can be applied as the speed-related target value. In this case, the control unit 170 (command generation unit 172) can generate a control command to reduce the difference between the current position of the end effector obtained using the visual sensor 310 and the position of the part, and to reduce the difference between the speed of the end effector and the speed target value. Here, the control unit 170 (target setting unit 176) can change the speed target value for each control cycle according to the difference between the current position of the end effector obtained using the visual sensor 310 and the position of the part.

In step 2 "Part gripping", "target reaction force" can be applied as the target measurement information, and "speed" can be applied as the speed-related value. In this case, the control unit 170 (command generation unit 172) can generate a control command to reduce the difference between the target reaction force and the current reaction force acting on the end effector obtained using the force sensor 320, and to reduce the difference between the end effector speed and the speed target value. Here, the control unit 170 (target setting unit 176) can change the speed target value for each control cycle according to the difference between the target reaction force and the reaction force acting on the end effector obtained using the force sensor 320.

In step 3 "Move the component towards the board", "board position" may be applied as the target measurement information, and "jerk" may be applied as the speed-related value. In this case, the control unit 170 (command generation unit 172) may generate a control command to reduce the difference between the current position of the end effector (or component) obtained using the visual sensor 310 and the position of the board, and to reduce the difference between the jerk of the end effector (or component) and the jerk target value. Here, the control unit 170 (target setting unit 176) may change the jerk target value for each control cycle according to the difference between the current position of the end effector (or component) obtained using the visual sensor 310 and the position of the board.

In step 4 "Approach", "difference in position between the end effector and the board" may be applied as the target measurement information, and "acceleration" may be applied as the speed-related value. In this case, the control unit 170 (command generation unit 172) may generate a control command to reduce the difference between the current position of the end effector (or component) obtained using the visual sensor 310 and the position of the board, and to reduce the difference between the acceleration of the end effector and the acceleration target value. Here, the control unit 170 (target setting unit 176) may change the acceleration target value for each control cycle according to the difference between the current position of the end effector (or component) obtained using the visual sensor 310 and the position of the board.

In step 5, "Board hole recognition," "Board hole" can be applied as the target measurement information. In this case, the control unit 170 (command generation unit 172) recognizes the board hole using the visual sensor 310.

In step 6 "Hole Approach", "Position of hole in board" and "Target reaction force" may be applied as target measurement information, and "Acceleration" may be applied as a speed-related value. In this case, the control unit 170 (command generation unit 172) may generate a control command to reduce the difference between the current position of the component obtained using the visual sensor 310 and the position of the hole in the board, and to reduce the difference between the acceleration of the end effector and the target acceleration value. The control unit 170 (command generation unit 172) may also generate a control command to reduce the difference between the current reaction force generated by the end effector obtained using the force sensor 320 and the target reaction force, and to reduce the difference between the acceleration of the end effector and the target acceleration value.

For each step from step 7 "hole tracing operation" to step 10 "grip release", a "target reaction force" for each step may be applied as target measurement information, and a "jerk" may be applied as a speed-related value. In this case, the control unit 170 (command generation unit 172) may generate a control command to reduce the difference between the current reaction force generated in the end effector obtained using the force sensor 320 and the target reaction force for each step, and to reduce the difference between the jerk of the end effector and the jerk target value for each step. Here, the control unit 170 (goal setting unit 176) may change the jerk target value for each control cycle according to the difference between the reaction force generated in the end effector obtained using the force sensor 320 and the target reaction force.

(5) Other embodiments The operation flows and operation examples in the above-described embodiments do not necessarily have to be executed in chronological order according to the order described in the flow diagram. For example, steps in the operations may be executed in an order different from that described in the flow diagram, or may be executed in parallel. In addition, some steps in the operations may be deleted, and additional steps may be added to the process.

A program may be provided that causes a computer to execute the operations according to the above-described embodiments. The program may be recorded on a computer-readable medium. Using the computer-readable medium, it is possible to install the program on a computer. Here, the computer-readable medium on which the program is recorded may be a non-transient storage medium. The non-transient storage medium is not particularly limited, and may be, for example, a storage medium such as a CD-ROM or a DVD-ROM.

As used in this disclosure, the terms "based on" and "in response to" do not mean "based only on" or "in response to" unless otherwise specified. The term "based on" means both "based only on" and "based at least in part on". Similarly, the term "in response to" means both "based only on" and "in response to" . In addition, the terms "include", "comprise", and variations thereof do not mean including only the items listed, but may include only the items listed, or may include additional items in addition to the items listed. In addition, the term "or" as used in this disclosure is not intended to be an exclusive or. In this disclosure, when articles are added by translation, such as a, an, and the in English, these articles are intended to include the plural unless the context clearly indicates otherwise.

The above describes the embodiment in detail with reference to the drawings, but the specific configuration is not limited to the above, and various design changes can be made without departing from the spirit of the invention.

(6) Supplementary Notes The following are additional notes regarding the features of the above-described embodiment.

・Note 1
an acquisition unit that acquires measurement information obtained by using a sensor for measuring an operating state of the robot device;
a control unit that repeatedly generates a control command for the robot device based on the measurement information,
The control unit derives a speed-related value related to the robot device from the measurement information or the control command, and generates the control command so as to reduce a difference between the speed-related value and a speed-related target value that is variable depending on an operating state of the robot device.

・Appendix 2
The control unit is
a derivation unit that derives the speed-related value from the measurement information or the control command;
a target setting unit that variably sets the speed-related target value in accordance with an operating state of the robot device;
and a command generating unit that generates the control command so as to reduce a difference between the speed-related target value and the speed-related value.

Appendix 3
The control device according to claim 1 or 2, wherein the speed-related value includes at least one of a speed, an acceleration, and a jerk.

Appendix 4
The control device according to any one of appendixes 1 to 3, wherein the control unit generates the control command to reduce a difference between the measurement information and target measurement information, and to reduce a difference between the speed-related value and the speed-related target value.

Appendix 5
the sensor includes a visual sensor;
The control device described in Appendix 4, wherein the control unit generates the control command to reduce a difference between visual measurement information obtained using the visual sensor and target visual measurement information, and to reduce a difference between the speed-related value and the speed-related target value.

Appendix 6
The sensor includes a force sensor,
The control device according to claim 4 or 5, wherein the control unit generates the control command to reduce a difference between force sense measurement information obtained using the force sensor and target force sense measurement information, and to reduce a difference between the velocity relation value and the velocity relation target value.

Appendix 7
The control device according to any one of claims 1 to 6, wherein the control unit changes the speed-related target value in accordance with a difference between the measurement information and target measurement information.

Appendix 8
the sensor includes a visual sensor;
The control device according to claim 7, wherein the control unit changes the speed-related target value in accordance with a difference between visual measurement information obtained using the visual sensor and target visual measurement information.

Appendix 9
The sensor includes a force sensor,
The control device according to claim 7 or 8, wherein the control unit changes the speed-related target value in accordance with a difference between force sense measurement information obtained using the force sensor and target force sense measurement information.

Appendix 10
The control device according to any one of appendixes 7 to 9, wherein the control unit selects one or more values to be used as the speed-related value from among speed, acceleration, and jerk, depending on a difference between the measurement information and target measurement information.

Appendix 11
The control unit is
a prediction unit that predicts a control result for a subsequent control operation performed on the robot apparatus, based on the control operation performed on the robot apparatus and a control result for the control operation;
and a correction unit that corrects the subsequent control content in accordance with the prediction.

Appendix 12
A library storage unit is further provided for storing a plurality of setting libraries prepared for each type of work,
each of the plurality of setting libraries includes setting information related to each of a series of steps;
The control device according to any one of claims 1 to 11, wherein the control unit performs the control using a setting library selected from the plurality of setting libraries according to a type of work to be actually performed.

Appendix 13
The control device according to claim 12, wherein each of the plurality of setting libraries includes setting information for the speed-related target value.

Appendix 14
A library storage unit is further provided for storing a plurality of recognition libraries prepared for each type of work,
Each of the plurality of recognition libraries includes a trained model used for image recognition processing of an object;
The control device according to any one of claims 1 to 13, wherein the control unit performs the image recognition process using a recognition library selected from the plurality of recognition libraries according to a type of task to be actually performed.

Appendix 15
acquiring measurement information obtained using a sensor for measuring an operating state of a robot device;
and repeatedly generating a control command for the robot device based on the measurement information.
The repeatedly generating the control command includes deriving a speed-related value for the robot device from the measurement information or the control command, and generating the control command so as to reduce a difference between the speed-related value and a speed-related target value that is variable depending on an operating state of the robot device.

Appendix 16
The control device includes:
acquiring measurement information obtained using a sensor for measuring an operating state of a robot device;
repeatedly generating a control command for the robot device based on the measurement information;
The repeatedly generating of the control command includes deriving a speed-related value for the robot device from the measurement information or the control command, and generating the control command so as to reduce a difference between the speed-related value and a speed-related target value that is variable depending on an operating state of the robot device.

100: Control device 101: Processor 102: Memory 103: External I/F
Description of the Reference Number 110: Motion generation unit 120: Acquisition unit 121: Image recognition unit 130: Data collection unit 140: Data storage unit 150: Setting acquisition unit 160: Library storage unit 170: Control unit 171A: Visual control unit 171B: Force/force control unit 172: Command generation unit 173: Weighting unit 174A: Prediction unit 174B: Correction unit 175: Derivation unit 176: Target setting unit 200: Robot device 210: Driving unit 211: First joint unit 212: Second joint unit 213: Third joint unit 214: Fourth joint unit 215: Fifth joint unit 216: Sixth joint unit 221: Base unit 222: Link 223: Link 224: Link 225: Link 226: Gripper 300: Sensor 310 : Visual sensor 320 : Force sensor 330 : Other sensor 400 : User I/F
410: Display device 420: Operation device 510: Transport device

Claims (17)

an acquisition unit that acquires measurement information obtained using a visual sensor or a force sensor for measuring an operating state of the robot device;
a control unit that repeatedly generates a control command for the robot device based on the measurement information,
The control unit derives a speed-related value related to the robot device from the measurement information or the control command, generates the control command so as to reduce a difference between the speed-related value and a speed-related target value that is variable depending on an operating state of the robot device , and changes the speed-related target value depending on a difference between the measurement information and target measurement information.
Control device. an acquisition unit that acquires measurement information obtained by using a sensor for measuring an operating state of the robot device;
a control unit that repeatedly generates a control command for the robot device based on the measurement information,
the control unit derives a speed-related value related to the robot apparatus from the measurement information or the control command, and generates the control command so as to reduce a difference between the speed-related value and a speed-related target value that is variable depending on an operating state of the robot apparatus;
The control unit selects one or more values to be used as the speed-related value from among speed, acceleration, and jerk, depending on a difference between the measurement information and the target measurement information.
Control device. The control unit is
a derivation unit that derives the speed-related value from the measurement information or the control command;
a target setting unit that variably sets the speed-related target value in accordance with an operating state of the robot device;
The control device according to claim 1 , further comprising: a command generating unit configured to generate the control command so as to reduce a difference between the speed-related target value and the speed-related value. The control device according to claim 1 or 2 , wherein the speed-related value includes at least one of a speed, an acceleration, and a jerk. The control device according to claim 1 , wherein the control unit generates the control command to reduce a difference between the measurement information and target measurement information, and to reduce a difference between the speed-related value and the speed-related target value. the sensor includes a visual sensor;
The control device according to claim 5, wherein the control unit generates the control command so as to reduce a difference between visual measurement information obtained using the visual sensor and target visual measurement information, and to reduce a difference between the speed relationship value and the speed relationship target value . The sensor includes a force sensor,
The control device according to claim 5 , wherein the control unit generates the control command to reduce a difference between force sense measurement information obtained using the force sensor and target force sense measurement information, and to reduce a difference between the speed relationship value and the speed relationship target value . the sensor includes a visual sensor;
The control device according to claim 1 or 2 , wherein the control unit changes the speed-related target value in accordance with a difference between visual measurement information obtained by using the visual sensor and target visual measurement information. The sensor includes a force sensor,
The control device according to claim 1 or 2 , wherein the control unit changes the speed-related target value in accordance with a difference between force sense measurement information obtained by using the force sensor and target force sense measurement information. The control unit is
a prediction unit that predicts a control result for a subsequent control operation performed on the robot apparatus, based on the control operation performed on the robot apparatus and a control result for the control operation;
The control device according to claim 1 , further comprising: a correction unit that corrects the subsequent control content in accordance with the prediction. a library storage unit that stores a plurality of setting libraries prepared for each type of work that the robot device can perform ;
each of the plurality of setting libraries includes setting information related to each of a series of steps;
The control device according to claim 1 or 2 , wherein the control unit controls the robot device using a setting library selected from the plurality of setting libraries according to a type of work to be actually performed. The control device according to claim 11 , wherein each of the plurality of setting libraries includes setting information for the speed-related target value. a library storage unit that stores a plurality of recognition libraries prepared for each type of work that the robot device can perform ,
Each of the plurality of recognition libraries includes a trained model used for image recognition processing of an object;
The control device according to claim 1 or 2 , wherein the control unit performs the image recognition process using a recognition library selected from the plurality of recognition libraries according to a type of an actual task to be performed. acquiring measurement information obtained using a visual sensor or a force sensor for measuring an operating state of a robot device;
and repeatedly generating a control command for the robot device based on the measurement information.
The control method includes repeatedly generating the control command, deriving a speed-related value for the robot device from the measurement information or the control command, generating the control command so as to reduce a difference between the speed-related value and a speed-related target value that is variable depending on an operating state of the robot device, and changing the speed-related target value depending on the difference between the measurement information and target measurement information . The control device includes:
acquiring measurement information obtained using a visual sensor or a force sensor for measuring an operating state of a robot device;
repeatedly generating a control command for the robot device based on the measurement information;
The program in which repeatedly generating the control command includes deriving a speed-related value for the robot device from the measurement information or the control command, generating the control command so as to reduce a difference between the speed-related value and a speed-related target value that is variable depending on an operating state of the robot device, and changing the speed-related target value depending on the difference between the measurement information and target measurement information . acquiring measurement information obtained using a sensor for measuring an operating state of a robot device;
and repeatedly generating a control command for the robot device based on the measurement information.
Repeatedly generating the control command includes deriving a speed-related value for the robot device from the measurement information or the control command, and generating the control command so as to reduce a difference between the speed-related value and a speed-related target value that is variable depending on an operating state of the robot device; and selecting one or more values to be used as the speed-related value from among speed, acceleration, and jerk depending on the difference between the measurement information and target measurement information.
Control methods. The control device includes:
acquiring measurement information obtained using a sensor for measuring an operating state of a robot device;
repeatedly generating a control command for the robot device based on the measurement information;
Repeatedly generating the control command includes deriving a speed-related value for the robot device from the measurement information or the control command, and generating the control command so as to reduce a difference between the speed-related value and a speed-related target value that is variable depending on an operating state of the robot device; and selecting one or more values to be used as the speed-related value from among speed, acceleration, and jerk depending on the difference between the measurement information and target measurement information.
Program.

Images