WO2017150073A1 - Work operation analysis system, work operation analysis method, and work operation analysis program - Google Patents

Abstract

This work operation analysis system is equipped with: production equipment (100), which is equipped with sensors (112), an equipment information generation unit (111) that generates equipment information including multiple timestamps identifying the time at which the sensors were activated, and a transmission unit (110) that transmits the equipment information to an analysis device; and the analysis device (102), which is equipped with a reception unit (120) that receives the equipment information, an analysis information buffer (133) that temporarily stores analysis information acquired from an imaging device, and an analysis unit (137) that waits until the analysis information, acquired at the respective times indicated by the multiple timestamps included in the received equipment information, has been stored in the analysis information buffer, and that analyzes the analysis information for a range that is specified using the times indicated by the multiple timestamps.

Description

Work motion analysis system, work motion analysis method, and work motion analysis program

The present invention relates to a technique for recognizing work operations.

In the field of assembly processing, workers are required to perform assembly processing of products according to the standard operation specified. Since the standard action is defined as the action necessary to maintain the quality of the product, if the worker performs an action (deviation action) that is different from the standard action, , The possibility of problems in quality increases. For this reason, a function (deviation action detection function) for automatically detecting a departure action from human motion information acquired using various sensors is required. When a deviating action is detected by the deviating action detecting function, the quality can be ensured by, for example, performing a re-inspection on the product that is the work target at that time or discarding the product itself.

As a method of automatically analyzing human movements, the invention described in Patent Document 1 discloses a method using moving images. In the method, first, feature data is calculated from a moving image, and the moving image is divided by finding a time-series change of the feature data, that is, a change in motion. In this method, time-series feature data or a symbol string representing time-series feature data is acquired from the divided moving images, and the motion is analyzed using them. This means that a complicated operation is divided into simpler operations, and there is an advantage that even a complicated operation can be analyzed.

In Patent Document 1, “feature data is calculated based on local motion information statistics, moving image data is divided from time-series changes of the data, and time-series feature data is calculated for each divided section”. Is disclosed.

JP2009-205282A

In the invention described in Patent Document 1, a moving image is divided by using a change in operation. However, the timing of starting and ending work and the change in operation are not necessarily the same timing, and thus the timing cannot be acquired. There is.

In order to solve the above-described problems, a work motion analysis system according to the present invention analyzes a device, a device information generation unit that generates device information including a plurality of time stamps that specify the time at which the sensor issued, and analyzes the device information. A production unit including a transmission unit that transmits to the apparatus, a reception unit that receives apparatus information, an analysis information buffer that temporarily stores analysis information acquired from the imaging apparatus, and a plurality of pieces of information included in the received apparatus information Analysis waits until the analysis information acquired at each time indicated by the time stamp is stored in the analysis information buffer, and analyzes the analysis information for the range specified using the time indicated by the multiple time stamps And an analysis device.

By acquiring the work start and / or end timings from the production equipment, it is possible to perform analysis only for the time zone during work, or at least the time zone that includes work.

1 is a configuration diagram of a work operation analysis system in Embodiment 1. FIG. 3 is a flowchart illustrating the operation of the production apparatus according to the first embodiment. 3 is a flowchart illustrating the operation of the analysis apparatus according to the first embodiment. It is the figure which showed the component of apparatus information. It is a structure of the data preserve | saved at the trigger generation table. It is the figure which showed the component of the trigger. It is the structure of the data preserve | saved at the information buffer for analysis. It is the structure of the data preserve | saved at the camera ID table. It is a 1st example of operation | movement of the information division part for analysis. It is a 2nd example of operation | movement of the information division part for analysis. It is a 3rd example of operation | movement of the information division part for analysis. It is the structure of the data preserve | saved at the operation | movement model storage part. It is the structure of the data preserve | saved at the log storage part. It is an example of a screen for notifying the analysis result displayed on a display part.

Hereinafter, examples will be described with reference to the drawings.

FIG. 1 is a system configuration diagram showing an overall image of a work motion analysis system. The work motion analysis system includes one or more production apparatuses 100 and an analysis apparatus 102.

The production apparatus 100 includes an apparatus information transmission unit 110, an apparatus information generation unit 111, one or more sensors 112, a control unit 113, and a production unit 114.

The analysis device 102 includes a device information reception unit 120, a trigger generation unit 121, a trigger generation table 123, a trigger buffer 131, an analysis information division unit 132, an analysis information buffer 133, a camera ID table 134, An operation model selection unit 135, an operation model storage unit 136, an analysis unit 137, a log storage unit 138, and a display unit 139 are included. One or more cameras 140 are connected to the analysis information buffer 133. A personal authentication device 150 and a scheduler 151 are connected to the analysis device 102.

The operation of the production apparatus 100 will be described later with reference to FIG. 2, and the operation of the analysis apparatus 102 will be described later with reference to FIG.

FIG. 2 is a flowchart showing the operation of the production apparatus 100. This flowchart will be described with reference to FIG.

S200 is processing in which the device information generation unit 111 acquires a sensor ID from the sensor 112. The sensor ID is information that can identify the sensor 112. This process makes it clear which sensor has issued the alarm. The sensor 112 is, for example, a contact sensor, and can acquire the timing when the worker picks up the product from the production apparatus or the timing when the product is introduced. The former means the work start timing of the worker, and the latter means the work end timing of the worker.

Further, the sensor 112 may be a start switch of the production unit 114. In that case, the control unit 113 receives the notification of the sensor 112 as the start switch, and controls the production unit 114 to perform production-related processing such as cutting, caulking, welding, and conveyance. In other words, the sensor 112 can acquire the timing when the worker inputs the product into the production apparatus and activates the production apparatus. This means the work end timing of the worker.

In S201, the apparatus information generation unit 111 adds at least a production apparatus ID that can identify the production apparatus and a time stamp that can identify the time to the sensor ID acquired in S200, and the apparatus information 400 shown in FIG. Is a process for generating The time stamp represents the timing at which the device information generation unit 111 detects the report from the sensor 112. Alternatively, if the sensor 112 includes a time-synchronizable clock, in S200, the device information generation unit 111 may receive the sensor ID and the timing issued by the sensor 112 together. In that case, in S201, the apparatus information generation unit 111 only needs to add the production apparatus ID to the received information.

S202 is a process in which the device information transmission unit 110 transmits the device information 400 to the analysis device 102. FIG. 3 is a flowchart showing the operation of the analysis apparatus 102. This flowchart will be described with reference to FIGS.

S300 is a process in which the device information receiving unit 120 receives the device information 400 from the device information transmitting unit 110. As shown in FIG. 5, the data 500 stored in the trigger generation table 123 is a production apparatus ID, a sensor ID, a work ID, and a work start / end flag, and these are associated with each other.

S301 is a process in which the trigger generation unit 121 reads the work ID and the work start / end flag corresponding to the sensor ID and the production apparatus ID included in the device information 400 with reference to the trigger generation table 123. As a result of this processing, a trigger 600 shown in FIG. 6 is generated, which includes at least those work IDs, work start / end flags, and a time stamp included in the device information. Hereinafter, the timing indicated by the time stamp included in the trigger 600 is simply referred to as the timing indicated by the trigger 600.

The work start / end flag indicates whether the timing indicated by the trigger 600 represents the work start or end timing. The former is called the start trigger and the latter is called the end trigger. Since there are two types of work processes where both start triggers and end triggers are generated (with pair triggers) and work processes where only start or end triggers are generated (without pair triggers), work start / end The flag also contains information about whether or not pair triggers are present.

Note that there may be no information in the trigger generation table 123 depending on the combination of the production device ID and the sensor ID. In this case, it means that the corresponding work does not exist, and the trigger 600 is not generated.

S310 is a process of temporarily storing the trigger 600 in the trigger buffer 131.

S 311 is a process of temporarily storing analysis information read from the camera 140 in the analysis information buffer 133. The analysis information stored here may be a moving image or information representing a posture of a person extracted from the moving image. Further, the posture information of a person may be position information of a part such as a person's hand, foot, shoulder, or head. In any case, as shown in FIG. 7, the data in the analysis information buffer 133 is stored as time-series information for each camera ID.

In FIG. 7, a simulation diagram in the case where position information of parts such as human hands, feet, shoulders, and heads is accumulated as time series information is shown as a simulation diagram 701 of analysis information.

S312 is a process in which the analysis information dividing unit 132 reads the trigger 600 stored in the trigger buffer 131. If the read trigger has a pair trigger, read it after both pair triggers are available. If the read trigger has no pair trigger, it is read as it is.

Thereafter, the analysis information dividing unit 132 refers to the camera ID table 134 and acquires the corresponding camera ID from the work ID included in the trigger 600.

As shown in FIG. 8, the data 800 in the camera ID table 134 is data in which a work ID and a camera ID are associated with each other.

S313 is processing in which the analysis information dividing unit 132 divides and reads analysis information from the analysis information buffer 133 according to the acquired camera ID and the time stamp included in the trigger. Hereinafter, the three patterns of division processing will be described with reference to FIGS.

The first pattern is when there is a pair trigger. In this case, the analysis information dividing unit 132 divides and reads analysis information in a range between both timings from the analysis information buffer 133 according to the timings indicated by the start trigger and the end trigger.

Referring to FIG. 9, the first pattern will be described. That is, in the simulation diagram 701 of the information for analysis, the first example 900 of the division target that is between the first example 901 of the timing indicated by the start trigger and the first example 902 of the timing indicated by the end trigger 900 Is shown.

The second pattern is when there is no pair trigger and the trigger is a start trigger. In this case, the range between the timing indicated by the start trigger and the timing after a predetermined time with respect to the timing indicated by the start trigger is divided and read.

The second pattern will be described with reference to FIG. That is, in the simulation diagram 701 of the information for analysis, a range between the second example 1001 of the timing indicated by the start trigger and the timing 1002 after a predetermined time from the second example of the timing indicated by the start trigger. A state in which the second example 1000 to be divided is divided is shown. Here, the predetermined range may be a value obtained by adding a margin to the standard work time of the corresponding work process.

The third pattern is when there is no pair trigger and the trigger is an end trigger. In this case, the range between the timing indicated by the end trigger and the timing indicated by the end trigger is divided and read.

FIG. 11 explains the third pattern. That is, in the simulation diagram 701 of the analysis information, the second example 1101 of the timing indicated by the end trigger and the range between the timing 1102 a predetermined time before the second example of the timing indicated by the end trigger. A mode that the 3rd example 1100 of a division | segmentation object is divided | segmented is shown. Here, the predetermined range may be a value obtained by adding a margin to the standard work time of the corresponding work process.

However, in any of the above three patterns, if the analysis information is not stored in time, it waits until the analysis information in the range to be read is stored.

S314 is a process in which the behavior model selection unit 135 selects a behavior model corresponding to the work ID from the behavior model storage unit 136 in accordance with the work ID included in the trigger 600. Alternatively, the operation model corresponding to the trigger work ID and the personal ID that can identify the person performing the corresponding work acquired from the personal authentication device 150 or the scheduler 151 may be selected. The personal authentication device 150 acquires the personal ID of the person who is performing the work corresponding to the work ID using the recognition device that is recognized by the biometric information or held by the person. The scheduler 151 has information on when each person performs a work in which work process, and specifies the personal ID of the person who is performing the work based on the work ID and time stamp included in the trigger 600. be able to.

As shown in FIG. 12, the data 1200 stored in the behavior model storage unit 136 is data in which a work ID, a personal ID, and a behavior model expressed in numerical values are associated with each other. When the personal ID is not used, an operation model is stored for each work ID. The motion model may represent human motion information as time-series information of motion vectors on an image or a representative value thereof, or may be time-series information about a human posture or a representative value thereof. Furthermore, it may be expressed as a probability distribution related to them. When expressing as a probability distribution, use a method that uses a Gaussian distribution or a Gaussian mixture distribution, which is a parametric method, a frequency distribution method that uses a nonparametric method, or a method that uses a Parzen window. can do.

S315 is a process in which the analysis unit 137 performs analysis using the analysis information divided in step S313 and the operation model selected in S314. Specifically, the degree of deviation representing how much the action indicated by the divided analysis information deviates from the selected action model is calculated and output as an analysis result. As a method of calculating the deviation degree, the dynamic programming method can be used if the selected behavior model is represented by time series information, and the Euclidean distance can be used if it is represented by a representative value. In addition, when the behavior model is expressed as a probability distribution, the Mahalanobis distance can be used if the probability distribution is a Gaussian distribution, and if it is expressed as another probability distribution, divided analysis information can be generated. The degree of deviation can be calculated by calculating the probability. Even when the behavior model is represented by a hidden Markov model, the probability that the divided analysis information can be generated can be calculated, and the deviation degree can be calculated.

The analysis result may be a plurality of values obtained by calculation using a plurality of methods. For example, if the degree of deviation is calculated by selectively using different parts of the human body such as the whole body, hand, foot, upper body, and lower body in the analysis information, an analysis result corresponding to each part can be obtained.

S316 is a process of accumulating the analysis result obtained in this way in the log accumulation unit 138. As shown in FIG. 13, data 1300 stored in the log storage unit is data in which a time stamp, a work ID, a personal ID, and an analysis result are associated with each other. Here, the time stamp is stored corresponding to the trigger 600 used in step S313. If the work start trigger is used, the time stamp (work start time stamp) is stored. If the work end trigger is used, the time stamp (work end time stamp) is stored. If both are used, save both.

S317 is a process in which the display unit 139 displays the time stamp, work ID, personal ID, and analysis result on the production apparatus 100. Here, the time stamp is stored corresponding to the trigger 600 used in step S313. If the work start trigger is used, the time stamp (work start time stamp) is displayed. If the work end trigger is used, the time stamp (work end time stamp) is displayed. If both are used, both are displayed.

FIG. 14 is a diagram illustrating an example 1400 of a screen displayed on the display screen 139. If any analysis result exceeds a predetermined threshold value, a message indicating that a deviating action has occurred is displayed as shown in FIG. 14, and the analysis result uses information for analysis of a specific part of the human body. If it is calculated in this way, it becomes easier to understand by simultaneously displaying the parts as shown in FIG. Thereafter, the worker or the supervisor who has seen the display screen 139 can deal with, for example, discarding the product when the deviation operation occurs or giving appropriate guidance to the worker who has performed the deviation operation. Become.

DESCRIPTION OF SYMBOLS 100 Production apparatus 102 Analysis apparatus 110 Apparatus information transmission part 111 Apparatus information generation part 112 Sensor 113 Control part 114 Production part 120 Apparatus information reception part 121 Trigger generation part 123 Trigger generation table 131 Trigger buffer 132 Analysis information division part 133 Information for analysis Buffer 134 Camera ID table 135 Operation model selection unit 136 Operation model storage unit 137 Analysis unit 138 Log storage unit 139 Display unit 140 Camera 150 Personal authentication device 151 Scheduler 400 Device information 600 Trigger

Claims (5)

A work motion analysis system comprising a production device and an analysis device,
The production apparatus includes a sensor, a device information generation unit that generates device information including a plurality of time stamps that specify times when the sensor has issued, and a transmission unit that transmits the device information to the analysis device. Prepared,
The analysis device includes a reception unit that receives the device information, an analysis information buffer that temporarily stores analysis information acquired from the imaging device, and an analysis unit.
The analysis unit waits until the analysis information acquired at each of the times indicated by the plurality of time stamps included in the received device information is accumulated in the analysis information buffer, and displays the times indicated by the plurality of time stamps. A work motion analysis system characterized in that the analysis information is analyzed for a range specified by use. The work motion analysis system according to claim 1,
The plurality of time stamps include a first time stamp indicating a work start time of the worker and a second time stamp indicating a work end time of the worker,
The work motion analysis system characterized in that the range is a range specified by the first time stamp and the second time stamp. The work motion analysis system according to claim 2,
The sensor includes a contact sensor and a start switch,
The first time stamp is a time stamp specifying the time when the contact sensor has issued, and the second time stamp is a time stamp specifying the time when the activation switch has issued. Characteristic work motion analysis system. A procedure for generating device information including a plurality of time stamps capable of specifying the time at which the sensor issued,
A procedure for temporarily storing analysis information acquired from the imaging device in an analysis information buffer;
A procedure of waiting until analysis information acquired at each of the times indicated by the plurality of time stamps is stored in the analysis information buffer;
And a procedure for analyzing the analysis information for a range specified using times indicated by the plurality of time stamps. A work motion analysis program for causing a computer to execute the work motion analysis method according to claim 4.

Images