WO2025047014A1 - Work improvement support system, work improvement support method, and program - Google Patents

Abstract

A work improvement support system (1) comprises: a work data generation unit (21) that generates work data by attaching, to a microscope image acquired by a microscope device (10) during the work of assembling or inspecting equipment using the microscope device (10), tag information relating to the work at the time of the acquisition of the microscope image; a quality information generation unit (22) that, on the basis of a plurality of sets of work data generated by the work data generation unit (21), generates quality information relating to the quality of the work for each work data classification classified by the tag information; and a display control unit (23) that displays the quality information generated by the quality information generation unit (22) on a display device (30).

Description

Work improvement support system, work improvement support method, and program

The disclosure of this specification relates to a work improvement support system, a work improvement support method, and a program, and in particular to a system, method, and program that supports the improvement of work performed under a microscope, such as the assembly of precision equipment.

Even today, when automation of work is progressing with the use of robots and other tools, there are still many products that require manual assembly. Medical equipment is one example. The assembly and inspection of precision equipment such as medical equipment requires detailed work, and is often done under a microscope, making the work highly difficult. Therefore, it is difficult to avoid variations in work efficiency and quality due to differences in the proficiency of workers.

Technology related to such technical issues is described, for example, in Patent Document 1. The work instruction system described in Patent Document 1 generates and displays work instruction information based on information entered by the worker and information pre-recorded in a database. By having the worker carry out the work while checking the displayed work instructions, errors in work procedures, etc. can be reduced.

Japanese Patent Application Publication No. 9-262745

However, simply displaying work instructions has limited effect on improving work. There is hope for the proposal of new technology that can further improve work efficiency and the variability in work quality that arises from the proficiency of workers when performing highly challenging work under a microscope.

In light of the above-mentioned circumstances, one aspect of the present invention aims to provide technology that improves the efficiency and quality variability of work performed under a microscope.

The work improvement support system according to one embodiment of the present invention is a work improvement support system that supports the improvement of work in assembling or inspecting equipment using a microscope device, and includes a work data generation unit that generates work data by attaching tag information related to the work at the time the microscope image was acquired to a microscope image acquired by the microscope device during the work of assembling or inspecting the equipment, a quality information generation unit that generates quality information related to the quality of the work for each classification of work data classified by the tag information based on multiple work data generated by the work data generation unit, and a display control unit that displays the quality information generated by the quality information generation unit on a display device.

In one aspect of the present invention, a work improvement support method includes a computer generating work data by attaching tag information related to the work performed at the time the microscopic image was acquired to a microscopic image acquired by a microscope device during the assembly or inspection of equipment using the microscope device, generating quality information, which is statistical information related to the quality of the work, for each classification of work data classified by the tag information based on the multiple generated work data, and displaying the generated quality information on a display device.

A program according to one aspect of the present invention causes a computer to execute the following processes: generate work data by attaching tag information related to the work performed when the microscopic image was acquired to a microscopic image acquired by a microscope device during the assembly or inspection of equipment using the microscope device; generate quality information, which is statistical information related to the quality of the work, for each classification of work data classified by the tag information based on the multiple generated work data; and display the generated quality information on a display device.

The above aspect makes it possible to improve the efficiency and quality variability of work performed under a microscope.

1 is a diagram illustrating a configuration of an operation improvement support system according to an embodiment. 11 is a diagram illustrating tag information that the work improvement support system adds to a microscope image. FIG. FIG. 2 is a diagram illustrating a flow of data processing performed by the operation improvement support system. FIG. 11 is a diagram for explaining assembly master information held by the operation improvement support system. FIG. 1 is a diagram illustrating a configuration of a microscope system according to an embodiment. FIG. 1 is a diagram illustrating an example of the configuration of an optical system included in a microscope system. FIG. 1 is a diagram illustrating an example of a slide set configuration. 1 is an example of an application screen displayed on a display device. FIG. 4 is a diagram showing an example of tag information. FIG. 11 is a diagram showing another example of tag information. FIG. 11 is a diagram showing an example of a means for selecting tag information. FIG. 11 is a diagram showing yet another example of tag information. 11 is another example of an application screen displayed on the display device. 11 is yet another example of an application screen displayed on the display device. 11 is yet another example of an application screen displayed on the display device. 11 is yet another example of an application screen displayed on the display device. 1 is an example of an image observed by looking through an eyepiece. 1 is another example of an image observed through an eyepiece. FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer for implementing a control device.

FIG. 1 is a diagram showing the configuration of a work improvement support system 1 according to one embodiment. The work improvement support system 1 is a system that supports the improvement of work performed using a microscope device 10, such as the assembly or inspection of precision equipment, such as medical equipment. However, the use of the work improvement support system 1 is not limited to precision equipment. It can be applied to any work performed using the microscope device 10.

The importance of work improvement efforts has long been recognized, and improvement activities consisting of the following steps are generally recommended at precision equipment manufacturing sites.
(Step 1) Find the defective part.
(Step 2) Analyze the defective part.
(Step 3) Use the analysis results to improve subsequent work.

In addition, images (including videos) have traditionally been collected as work records necessary for the above improvement activities. However, in the past, in many cases, images collected as work records were simply stored on a server and managed in a way that made them extremely difficult to search. This makes it difficult to find defects from work records after the fact, and the reality is that it is difficult to effectively carry out improvement activities that make use of work records.

The Work Improvement Support System 1 was devised in light of this situation, and in addition to tagging work records so that they can be used effectively, it automatically classifies and analyzes the work records, and creates and displays quality information that contributes to work improvement for each classification. This makes it possible to improve the variation in work quality and work efficiency caused by the proficiency of workers, even in the manufacture of precision equipment, which requires highly difficult work.

FIG. 2 is a diagram explaining tag information 50 that the work improvement support system 1 attaches to a microscopic image 40. FIG. 3 is a diagram explaining the flow of data processing performed by the work improvement support system 1. FIG. 4 is a diagram explaining assembly master information 90 held by the work improvement support system 1. The configuration and operation of the work improvement support system 1 will be explained below with reference to FIGS. 1 to 4.

As shown in FIG. 1, the work improvement support system 1 includes an information processing device 20 that processes microscopic images acquired by a microscope device 10 during the assembly or inspection of a precision device. The work improvement support system 1 may further include a microscope device 10 and a display device 30.

The microscope device 10 is equipped with an imaging device and acquires a microscope image 40 during the assembly or inspection of a precision instrument. The microscope device 10 may be, for example, a binocular stereo microscope, a metal microscope, a measuring microscope, a digital microscope, a laser microscope, or an AR (Augmented Reality) microscope, which will be described later.

The display device 30 is, for example, a display device connected to the information processing device 20, a display device in a projector equipped in an AR microscope, or a display device of another information processing device (for example, a tablet terminal or smart glasses) that communicates with the information processing device 20.

As shown in FIG. 1, the information processing device 20 includes a work data generation unit 21, a quality information generation unit 22, and a display control unit 23.

In the information processing device 20, the work data generation unit 21 generates work data 60 by attaching tag information 50 related to the work at the time the microscopic image 40 was acquired to the microscopic image 40 acquired by the microscope device 10 during the work of assembling or inspecting a precision instrument, as shown in Figures 2 and 3. The microscopic image 40 to which the tag information 50 is attached by the work data generation unit 21 may be a still image or a moving image. As the work data generation unit 21 repeatedly generates the work data 60, multiple work data 60 are accumulated in the information processing device 20 (step S1). Note that hereinafter, a collection of work data 60 will be referred to as a work data set 70.

The tag information 50 attached to the microscopic image 40 by the work data generation unit 21 is used at least to classify the work data. As shown in FIG. 2, the tag information 50 desirably includes work content information 51 that identifies the work performed when the microscopic image 40 was acquired, and work result information 53 that indicates an evaluation of the results of the work performed when the microscopic image 40 was acquired. As shown in FIG. 2, the tag information 50 desirably further includes work condition information 52 that identifies the conditions of the work performed when the microscopic image 40 was acquired.

The hierarchical structure of the multiple assemblies that make up the precision equipment, and the work and procedures for assembling and manufacturing each assembly are known in advance and are managed as assembly master information 90 shown in FIG. 3 in the information processing device 20. The work content information 51 may be information that identifies the work at the time the microscopic image 40 was acquired by referring to the assembly master information 90. The work content information 51 may include, for example, work type information that identifies the type of work R1 at the time the microscopic image 40 was acquired, work procedure information that identifies the work procedure R2 at the time the microscopic image 40 was acquired, and work stage information that identifies the work stage R3 at the time the microscopic image 40 was acquired, as shown in FIGS. 2 and 4, respectively.

The work condition information 52 is information that identifies the worker, the equipment, the specimen, and other constraints as conditions under which the work is performed. For example, as shown in FIG. 2, the work condition information 52 may include worker information that identifies the worker, procedure manual information that identifies the procedure manual that the worker refers to when working, camera setting information such as the gain of the imaging device and the exposure time, microscope status information such as the microscopy method, magnification, and illumination light amount, specimen information such as the specimen serial number and lot number, specimen status information such as the specimen posture, time information such as the allowable work time, information on the start and end times of work (for example, 9:00 AM, 11:00 AM, etc.), and any other text information.

In addition, the work result information 53 is information that qualitatively or quantitatively evaluates the results of the work and information that serves as the basis for that evaluation, and may include, for example, as shown in FIG. 2, information on the time required for the work, the time (including date and time) when the work started and the time (including date and time) when the work ended (for example, 9:00 AM and 11:00 AM on May 1, 2024), information indicating whether the work was good or bad, information that serves as the basis for judging whether the work was good or bad, information on measured work results, and information indicating the quality of the product that is the final result of the work.

The timing at which the work data generating unit 21 performs the process of attaching tag information 50 is not particularly limited. For example, the work data generating unit 21 may attach tag information 50 each time a microscopic image 40 is acquired. Furthermore, the work data generating unit 21 may attach tag information 50 periodically or irregularly to one or more microscopic images 40 acquired and stored by the microscope device 10. Furthermore, the tag information 50 attached to the microscopic image 40 may be changed later. For example, after a worker attaches tag information 50, an administrator may update and change the tag information 50.

The work data generating unit 21 may generate tag information 50 to be attached to the microscopic image 40 based on an input operation by the worker, and attach the generated tag information 50 to the microscopic image 40. The work data generating unit 21 may also automatically generate tag information 50 to be attached to the microscopic image 40 based on information such as the analysis results of the microscopic image 40 and the state of the microscope device 10 and the information processing device 20, and attach the generated tag information 50 to the microscopic image 40.

As shown in FIG. 3, the quality information generating unit 22 generates quality information 80 related to the quality of the work for each classification of the work data 60 classified by the tag information 50 based on the multiple work data (work data set 70) generated by the work data generating unit 21. Furthermore, the display control unit 23 causes the display device 30 to display the quality information 80 generated by the work data generating unit 21.

More specifically, as shown in FIG. 3, the quality information generation unit 22 classifies the working dataset 70 into a plurality of working datasets 70 (working dataset 71, working dataset 72, working dataset 73, working dataset 74) based on the tag information 50 (step S2), and generates quality information 80 for each classified working dataset (step S3).

It is desirable that the quality information generating unit 22 analyzes the multiple pieces of work result information 53 included in the multiple pieces of work data 60, and generates quality information 80 for each classification based on the multiple pieces of work content information 51 included in the multiple pieces of work data 60. For example, the quality information generating unit 22 may classify the work data 60 into each process of precision equipment assembly work based on the work content information 51, and generate quality information 80 for each process.

The quality information generating unit 22 may also analyze the multiple pieces of work result information 53 included in the multiple pieces of work data 60, and generate quality information 80 for each classification based on the multiple pieces of work content information 51 and the multiple pieces of work condition information 52 included in the multiple pieces of work data 60. For example, the quality information generating unit 22 may classify the work data 60 into combinations of processes and workers in the assembly work of precision equipment, based on the work content information 51 and the work condition information 52, and generate quality information 80 for each combination of process and worker.

The timing at which the quality information generating unit 22 performs the process of generating the quality information 80 is not particularly limited. The process of generating the quality information 80 may be performed before the display control unit 23 causes the display device 30 to display the quality information 80. The quality information generating unit 22 may generate the quality information 80, for example, in synchronization with the process of displaying the quality information 80. Furthermore, the quality information generating unit 22 may generate the quality information 80 periodically or irregularly, for example, independent of the process of displaying the quality information 80.

The timing at which the display control unit 23 performs the process of displaying the quality information 80 on the display device 30 is not particularly limited. For example, the display control unit 23 may cause the display device 30 to display the quality information 80 of the classification corresponding to the current work during the work of assembling or inspecting a precision device using the microscope device 10. The display control unit 23 may also cause the display device 30 to display the quality information 80 outside the work period of assembling or inspecting a precision device. When displaying the quality information 80 on the display device 30 outside the work period of assembling or inspecting a precision device, the display control unit 23 may, for example, cause the display device 30 to display the quality information 80 of the classification corresponding to the work identified based on the input information in response to an input operation by a worker using an input device.

As described above, in the work improvement support system 1, work records are managed as work data 60 in which work-related tag information 50 is attached to microscopic images 40. The accumulated work records, or work data 60 (work data set 70), are then automatically classified based on the tag information, and quality information 80 is generated and displayed for each classification.

Therefore, with the work improvement support system 1, it is not just about storing data; since the process from storing data to generating information and displaying the information (feedback) is designed in advance, information that contributes to work improvement from work records can be easily fed back to relevant parties such as workers and managers. As a result, it is possible to improve the efficiency and quality variation of work performed under a microscope.

The work improvement support system 1 also classifies the work data 60 by the work content information 51, and generates quality information 80 from the work result information 53 attached to the work data 60 (microscope image 40) for each classification. This makes it possible to provide feedback to the relevant parties on information according to the work process and the target of the work. This allows the relevant parties to specifically recognize the points that need improvement for each task, making it easier to improve the work.

The work improvement support system 1 also classifies the work data 60 by work content information 51 and work condition information 52, and generates quality information 80 from the work result information 53 attached to the work data 60 (microscope image 40) for each classification. This makes it possible to provide feedback to the relevant parties on information according to the work conditions in addition to the work process and work target. This allows the relevant parties to specifically recognize points that need improvement for individual tasks under specific work conditions, making it even easier to improve the work.

In addition, the work improvement support system 1 displays quality information 80 of the classification corresponding to the current work on the display device 30 during the assembly or inspection of precision equipment. This allows the worker to check the quality information 80 related to the work while working. Therefore, the work improvement support system 1 can more reliably improve the efficiency and quality of work.

The work improvement support system 1 also displays on the display device 30 quality information 80 of the classification corresponding to the work identified based on information input using the input device. This makes it possible to obtain information even outside the work period. Therefore, the work improvement support system 1 can provide necessary information not only to workers but also to other related parties such as managers.

Although not specifically mentioned, the work data and the various data that constitute it (microscope images, tag information, quality information) may be recorded in the recording device of the information processing device 20, or may be recorded in other storage devices (e.g., storage of a dedicated database server, storage on the cloud).

Below, we will explain the configuration of a microscope system 2 in which the work improvement support system 1 is applied to an AR microscope. Fig. 5 is a diagram illustrating the configuration of a microscope system 2 according to one embodiment. Fig. 6 is a diagram illustrating the configuration of an optical system 110 included in the microscope system 2. Fig. 7 is a diagram illustrating the configuration of a slide set 600.

The microscope system 2 shown in FIG. 5 uses a prepared slide set to provide the worker with appropriate information at the necessary timing while working under a microscope, and is an example of the work improvement support system described above. The configuration of the microscope system 2 will be described with reference to FIG. 5 to FIG. 7.

The microscope system 2 includes a microscope device 100, a control device 200, a display device 300, a number of input devices 400 (a mouse 401, a keyboard 402, a foot switch 403, a barcode reader 404), and a web camera 500.

The microscope device 100 is an example of the microscope device 10 described above. The microscope device 100 is a stereo microscope capable of stereoscopically viewing a sample, and is an AR microscope that projects the above-mentioned quality information 80 onto an image plane where an optical image is formed. The microscope device 100 is equipped with a microscope optical system 110 shown in FIG. 6.

The microscope optical system 110 is an optical system for a stereo microscope. A user can observe the optical image formed by the microscope optical system 110 on the object side of the eyepiece lens 106 (eyepiece lens 106a, eyepiece lens 106b) with both eyes via the eyepiece lens 106, and can observe the sample in three dimensions. For this reason, the microscope device 100 is suitable for applications such as the assembly of precision equipment.

The microscope device 100 is equipped with a zoom lens 102 (zoom lens 102a, zoom lens 102b) that can be operated with a zoom handle 130. By operating the zoom handle 130, the observation magnification can be changed while continuing to observe the sample by looking through the eyepiece lens 106.

The microscope device 100 is equipped with a focusing handle 140. By manipulating the focusing handle 140, the distance between the sample and the objective lens 101 can be changed to focus on the sample.

The microscope device 100 is equipped with an imaging device 112 that captures an image of a sample and acquires a microscopic image of the sample. The eyepiece tube 120 to which the eyepiece lens 106 is attached is a triplet tube, and the imaging device 112 is attached to the eyepiece tube 120. The imaging device 112 is provided with a two-dimensional image sensor. The image sensor is not particularly limited, but may be, for example, a CCD image sensor or a CMOS image sensor. The microscope image 40 acquired by the imaging device 112 is output to the control device 200. The microscope image 40 may also be output directly to the display device 300.

As shown in FIG. 6, light split by a beam splitter 103a, such as a half mirror, from one of the left and right optical paths of the microscope optical system 110 enters the imaging device 112 via an imaging lens 111. Note that an ND prism 103b is provided in the other optical path to compensate for the optical path length of the left and right optical paths caused by the beam splitter 103a and to suppress the difference in light quantity.

The microscope device 100 is equipped with a projector 113 that projects information onto an image plane where the imaging lens 105 (imaging lens 105a, imaging lens 105b) forms an optical image. The projector 113 is a device that projects and superimposes information onto the image plane in accordance with commands from the control device 200. More specifically, the projector 113 superimposes, for example, the above-mentioned quality information 80 onto the image plane. Note that the type of projector 113 is not particularly limited. The projector 113 may be configured using a display device such as a liquid crystal device or a digital mirror device. The projector 113 is an example of the above-mentioned display device 30.

Projector 113 is provided inside eyepiece tube 120. Light from projector 113 passes through projection lens 114 and multiple beam splitters (beam splitter 115, beam splitter 104a, beam splitter 104b) and is guided to the left and right optical paths of microscope optical system 110.

The eyepiece tube 120 is provided with an operation unit 121 as shown in FIG. 5. By operating the operation unit 121, the user can switch the projector 113 ON/OFF and instruct it to start or stop superimposing information on the image surface.

The control device 200 is a device that controls the microscope device 100 and the display device 300, and is an example of the information processing device 20 described above. The control device 200 generates application screen data during the assembly or inspection work of a precision device, and displays the application screen on the display device 300 based on the screen data. The screen data is generated using a slide set 600, exemplified in FIG. 7, which is stored in advance in the control device 200.

Slide set 600 is made up of multiple slides (slide set 601, slide set 602, slide set 603, slide set 604, slide set 605, slide set 606). Slide set 600 is, for example, information that supports the assembly or inspection of precision equipment, and more specifically, corresponds to a procedure manual for assembly or inspection work. The order of the multiple slides is also predetermined. In other words, slide set 600 is made up of multiple slides that have been ordered in advance.

Each of the multiple slides contains information on each process of assembling or inspecting a precision device, and more specifically, the information includes the work content at each process.

The control device 200 generates screen data based on information about a target slide selected from among the multiple slides included in the slide set 600. By the control device 200 appropriately switching the target slide, the microscope system 2 can switch the application screen displayed on the display device 300 to provide the user with appropriate information according to the work process.

The control device 200, like the information processing device 20, also includes a work data generation unit 21, a quality information generation unit 22, and a display control unit 23. The control device 200 causes the display device 300 and the display device of the projector 113 to display the quality information 80.

Specifically, the control device 200 controls the display device 300 so that quality information 80 is displayed on an application screen displayed on the display device 300 during the assembly or inspection of the precision equipment. Furthermore, the control device 200 outputs the quality information 80 to the projector 113 and causes the display device of the projector 113 to display the quality information 80 during the assembly or inspection of the precision equipment. As a result, the quality information 80 is superimposed on the image plane by the projector 113, so that a user of the microscope device 100 can look through the eyepiece 106 to confirm the quality information 80 displayed superimposed on the optical image.

The display device 300 and the input device 400 are connected to the control device 200. The display device 300 is, for example, a liquid crystal display or an organic EL display, and is an example of the display device 30 described above. The web camera 500 transmits captured images to the control device 200 via a network such as the Internet. The web camera 500 captures, for example, an image of a user using the microscope system 2.

FIG. 8 is an example of an application screen displayed on the display device. FIG. 9 is a diagram showing an example of tag information. FIG. 10 is a diagram showing another example of tag information. FIG. 11 is a diagram showing an example of a means for selecting tag information. FIG. 12 is a diagram showing yet another example of tag information. Below, the process of generating work data performed by the microscope system 2 during assembly work will be specifically described with reference to FIGS. 8 to 12.

The control device 200 executes a software application (hereinafter referred to as a work support application) that supports assembly or inspection work performed under a microscope. As a result, an application screen corresponding to the work process based on the procedure manual is displayed on the display device 300. An instruction to switch the work process can be issued, for example, by pressing a switch provided on the zoom handle 130 or by operating the mouse 401 or foot switch 403. Note that an instruction to switch the work process may be issued based on the results of analysis of a microscope image (live image), and artificial intelligence may be used for image analysis.

Window 301 shown in FIG. 8 is an example of a screen displayed during the work of attaching chip resistors for assembly C1. Window 301 includes a display area 305 for selecting a slide, a display area 310 for displaying the work procedure, a display area 320 for displaying the latest microscope image acquired by microscope device 100, a display area 330 for displaying an explanation of the work procedure, an input area 340 for inputting the work results, and a button 350 for inputting image capture instructions.

While performing chip resistor installation work using the microscope device 100, the worker can check the window 301 displayed on the display device 300 to confirm the work procedure for the chip resistor installation work, the current process (stage) in that procedure, and the content of the work to be done in that process.

When a worker wishes to leave a work record for the current task, he or she simply selects any option from those displayed in input area 340 and then presses button 350. When button 350 is pressed, control device 200 acquires microscope image M1, attaches tag information to microscope image M1, and generates and records work data.

The tag information includes information corresponding to the option selected in the input area 340 as work result information. In this example, work result information indicating the success of the work is included. The tag information also includes work content information that identifies the work. In this example, work content information indicating that the work type is chip resistor mounting work and that the work stage is the stage of attaching pre-solder to the land is included. Information indicating the work type and work stage may be information on the slide being displayed on the display device 300, that is, the ID and page number of the work procedure manual. Unlike the work result information, the work content information can be automatically generated based on, for example, the information displayed in the display area 305, the display area 310, and the display area 330, without being input by the worker. Furthermore, work stage information may be automatically and retroactively generated based on information on the work time corresponding to the microscope image 40 from information on what work was being performed at each time or information on the work that should be performed at each time. Furthermore, the work content may be inferred from the time, not just the factual information on what work was performed.

The work result information included in the tag information may further include work time information T1 as shown in FIG. 9. The work time information T1 may be calculated from information on the work start time and the work end time. Furthermore, the work result information included in the tag information may also include annotation information T2 shown in FIG. 10 that the worker affixed to the microscope image as the basis for the work evaluation result entered in the input area 340.

Also, when an input selecting "NG" is made in input area 340, the options in input area 340 may be switched to options 341 shown in FIG. 11. Information corresponding to one or more options selected from options 341 may be included in the tag information as work result information indicating the basis for the work evaluation result.

Furthermore, the control device 200 may analyze the microscope image to measure the object in the image, and may include the measurement result in the tag information as work result information. For example, as shown in FIG. 12, the control device 200 may analyze the microscope image M2 acquired after the work is completed to measure the size of the solder area, and may include the measurement result in the tag information as work result information. The control device 200 may also determine whether the measurement result is within a preset tolerance range, and include the determination result in the tag information as work result information. That is, the control device 200 may automatically generate the work result information.

Furthermore, the tag information may include working condition information. The working condition information may include information for identifying the worker, such as information entered from the keyboard 402 when launching the work support application and logging in, or information obtained by reading the barcode on the worker's name tag with the barcode reader 404. The working condition information may also include sample information read from an RFID tag attached to a sample or container. The working condition information may also include information for identifying information provided to the user during work, such as ID information of a procedure manual selected in the work support application. The working condition information may also include settings of the microscope device 100 (e.g., magnification, type of objective lens, gain, exposure time, illumination light intensity), etc.

In this way, in the microscope system 2, when an operator records a microscope image as a work record, tag information is automatically or manually attached to the microscope image to generate work data. Furthermore, the generated work data is classified based on the work content information contained in the tag information, and, for example, quality information is generated based on the work result information for each work process.

The quality information may be, for example, statistical information of the option selected in the input area 340 (e.g., the respective proportions of OK, NG, and ambiguous). The quality information may also be statistical information of an option selected as the basis for an NG evaluation from among the options 341 (e.g., ranking information of the NG reasons). The quality information may also be information obtained by statistically processing measurement values such as the work time and the size of a specified area (e.g., average value). The following analysis may also be performed using the work time information.
a) Trend information on the quality of work at different work times (night or morning) b) Trend information by season c) Sorting of work result information

Each of Figs. 13 to 16 is another example of an application screen displayed on the display device 300. Below, the display process of quality information performed by the microscope system 2 during assembly work will be specifically described with reference to Figs. 13 to 16.

Window 301 shown in FIG. 13 is an example of a screen displayed during the work of attaching chip resistors for assembly C1, and is similar to the example screen shown in FIG. 8, except that quality information Q1 and button 360 are displayed.

The control device 200 may display quality information Q1 in an application screen corresponding to the work process, as shown in FIG. 13. The control device 200 identifies the current work based on the information displayed in the display area 305, the display area 310, and the display area 330, obtains quality information Q1 for the classification corresponding to the identified work, and causes the display device 300 to display the quality information Q1. In this example, the quality information Q1 is ranking information for defects in the current work (in this example, the work of installing chip resistors).

In FIG. 13, the quality information is displayed on the latest microscope image M1 (i.e., a live image) acquired by the microscope device 100, but the display position of the quality information is not limited to on the microscope image M1. Also, while FIG. 13 shows an example in which quality information in a work process is displayed regardless of the number of occurrences of defects, their frequency, the magnitude of their impact, etc., whether or not to display the quality information may be determined based on the number of occurrences of defects, their frequency, their impact, and the current time.

If the worker wishes to check the quality information in more detail, he or she may press button 360 to check additional quality information. FIG. 14 shows how window 302 pops up when button 360 is pressed. Window 302 contains quality information (quality information Q2, quality information Q3) that is different from quality information Q1.

Quality information Q2 is an example of information calculated by performing statistical calculations on work result information, and includes average work time, first-run rate, and process capability index (Cp, Cpk). Quality information Q3 is an example of information that graphically displays the statistical results of work result information, and is a graph showing the frequency distribution of work times.

13 shows an example in which the microscope system 2 automatically displays quality information during assembly work, but the quality information may also be displayed when an operator explicitly requests display, as shown in FIG. 15. The control device 200 may display quality information Q4 of a classification corresponding to the specified image in the display area 380 by specifying an image using the input area 371 shown in FIG. 15. The control device 200 may also display quality information Q4 of a classification related to the keyword in the display area 380 by inputting a keyword in the input area 372 shown in FIG. 15. Here, a keyword is any word including tag information. Also, by inputting a date and time, time zone, etc. instead of a keyword, quality information Q4 of a classification related to the input date and time zone may be displayed.

Note that the information searched for by input into input areas 371 and 372 is not limited to quality information. The control device 200 may identify related work from the input into input areas 371 and 372, and display in the display area 380, for example, a good product image M3 (also called a model image or reference image) that has been recorded in advance in association with the work, as shown in FIG. 16. The control device 200 may also display the good product image M3 together with the quality information Q4.

As described above, in the microscope system 2, quality information related to the current task is displayed during assembly work, thereby alerting the worker. This makes it possible to improve the quality and efficiency of the work. Also, as shown in FIG. 13, by automatically displaying quality information on the application screen according to the task, the worker can be semi-forced to check the quality information. Also, as shown in FIG. 15, by displaying quality information according to the request of the worker as shown in FIG. 14 and FIG. 15, unnecessary displays can be reduced, and excessive disruption of concentration on the task can be avoided. Also, as shown in FIG. 16, a reference image may be displayed instead of or in addition to the quality information.

17 is an example of an image observed by looking through the eyepiece 106. Although FIGS. 13 to 15 show an example in which the microscope system 2 displays quality information on the display device 300 during assembly work, the quality information may be displayed on the display device of the projector 113. The control device 200 causes the display device of the projector 113 to display the quality information Q1, so that the quality information Q1 is projected and superimposed on the image M4 (optical image) of the specimen, as shown in FIG. 17. This allows the worker to check the quality information Q1 while looking through the eyepiece 106 and observing the specimen. In other words, the quality information Q1 can be checked while continuing work under the microscope device 10, so that the quality of work, etc. can be improved without sacrificing the efficiency of operation. Furthermore, the control device 200 may cause the display device of the projector 113 to display information U1 explaining the work procedure in addition to the quality information Q1, as shown in FIG. 17. This allows the worker to check the quality information Q1 and the information U1 explaining the work procedure while looking through the eyepiece 106 and observing the specimen. Therefore, it is possible to simultaneously improve the efficiency and quality of work while continuing to work under the microscope device 10.

FIG. 18 is another example of an image observed by looking through the eyepiece 106. Although FIG. 17 shows an example in which quality information and other information are displayed on an optical image, this information does not necessarily have to be displayed on the optical image. As shown in FIG. 18, in a state in which light passing through the objective lens 101 is blocked by a light blocking means arranged on the optical path of the microscope optical system 110, the control device 200 may project an image onto the image plane using light from the display device of the projector 113. The image projected onto the image plane may be an image M5 of a menu screen or an image M6 of a search screen, and in response to an operation performed by the operator while these images are displayed, an image M7 including quality information and other information may be projected onto the image plane and made observable via the eyepiece 106.

FIG. 19 is a diagram illustrating an example of the hardware configuration of a computer 200a for realizing the control device 200 according to the embodiment described above. The hardware configuration shown in FIG. 19 includes, for example, a processor 201, a memory 202, a storage device 203, a reading device 204, a communication interface 206, and an input/output interface 207. The processor 201, the memory 202, the storage device 203, the reading device 204, the communication interface 206, and the input/output interface 207 are connected to each other, for example, via a bus 208.

The processor 201 may be, for example, a single processor, a multiprocessor, or a multicore processor. The processor 201 functions as a work data generation unit 21, a quality information generation unit 22, and a display control unit 23 by reading and executing programs stored in the storage device 203.

Memory 202 may be, for example, a semiconductor memory and may include a RAM area and a ROM area. Storage device 203 may be, for example, a semiconductor memory such as a hard disk or a flash memory, or an external storage device.

The reader 204 accesses the removable storage medium 205 according to instructions from the processor 201, for example. The removable storage medium 205 is realized by, for example, a semiconductor device, a medium where information is input and output by magnetic action, or a medium where information is input and output by optical action. An example of a semiconductor device is a USB (Universal Serial Bus) memory. An example of a medium where information is input and output by magnetic action is a magnetic disk. An example of a medium where information is input and output by optical action is a CD (Compact Disc)-ROM, a DVD (Digital Versatile Disc), a Blu-ray Disc, etc. (Blu-ray is a registered trademark).

The communication interface 206 communicates with other devices (e.g., the microscope device 100, the Web camera 500, etc.) according to instructions from the processor 201, for example. The input/output interface 207 is, for example, an interface between the input device 400 and the output device. The input device 400 is, for example, a device such as a mouse 401, a keyboard 402, or a foot switch 403 that accepts instructions from a user. The output device is, for example, the display device 300, and an audio device such as a speaker.

The program executed by the processor 201 is provided to the computer in the following form, for example.
(1) It is pre-installed in the storage device 203.
(2) Provided by a removable storage medium 205.
(3) Provided from a server such as a program server.

Note that the hardware configuration of the computer for realizing the control device described with reference to FIG. 19 is an example, and the embodiment is not limited to this. For example, part of the above-mentioned configuration may be deleted, or new configuration may be added. In another embodiment, for example, some or all of the functions of the above-mentioned processing device may be implemented as hardware using an FPGA (Field Programmable Gate Array), SoC (System-on-a-Chip), ASIC (Application Specific Integrated Circuit), and PLD (Programmable Logic Device).

The above-described embodiments are illustrative examples to facilitate understanding of the invention, and the present invention is not limited to these embodiments. Modifications of the above-described embodiments and alternative forms to the above-described embodiments may be included. In other words, the components of each embodiment may be modified without departing from the spirit and scope of the embodiment. New embodiments can be implemented by appropriately combining multiple components disclosed in one or more embodiments. Some components may be deleted from the components shown in each embodiment, or some components may be added to the components shown in the embodiments. Furthermore, the order of the processing steps shown in each embodiment may be changed as long as there is no contradiction.

In this specification, the expression "based on A" does not mean "based only on A," but rather means "based at least on A," and further means "based at least partially on A." In other words, "based on A" may be based on B in addition to A, or may be based on a part of A.

1: Work improvement support system 2: Microscope system 10, 100: Microscope device 20: Information processing device 21: Work data generation unit 22: Quality information generation unit 23: Display control unit 30, 300: Display device 40: Microscope image 50: Tag information 51: Work content information 52: Work condition information 53: Work result information 60: Work data 70, 71, 72, 73, 74: Work data set 80, Q1, Q2, Q3, Q4: Quality information 90: Assembly master information 106, 106a, 106b: Eyepiece lens 110: Microscope optical system 112: Imaging device 113: Projector 200: Control device 200a: Computer 201: Processor

Claims (8)

a task data generating unit that generates task data by attaching tag information related to the task at the time of acquisition of a microscope image acquired by the microscope device during an operation of assembling or inspecting a device using the microscope device;
a quality information generation unit that generates quality information related to a quality of the work for each classification of the work data classified by the tag information based on the plurality of work data generated by the work data generation unit;
a display control unit that displays the quality information generated by the quality information generation unit on a display device. 2. The work improvement support system according to claim 1,
The tag information is
Work content information that identifies the work performed when the microscopic image was acquired; and
and work result information indicating an evaluation of the result of the work at the time of acquiring the microscopic image,
The quality information generating unit
A work improvement support system characterized by analyzing a plurality of pieces of work result information contained in the plurality of work data, and generating the quality information for each of the classifications classified based on a plurality of pieces of work content information contained in the plurality of work data. 3. The work improvement support system according to claim 2,
The tag information further includes work condition information that identifies the conditions of the work at the time of acquiring the microscopic image,
The quality information generation unit analyzes the plurality of pieces of work result information and generates the quality information for each of the classifications based on the plurality of pieces of work content information and the plurality of pieces of work condition information contained in the plurality of work data. 3. The work improvement support system according to claim 1,
The operation improvement support system is characterized in that the display control unit causes the display device to display quality information of a classification corresponding to a current operation during the operation of assembling or inspecting the device. The work improvement support system according to claim 4, further comprising:
The microscope device includes:
The microscope device includes:
an imaging device for imaging the sample;
a microscope optical system including an eyepiece and forming an optical image of the sample on an object side of the eyepiece;
and a display device that superimposes the quality information on an image plane on which the optical image is formed. 3. The work improvement support system according to claim 1,
The operation improvement support system is characterized in that the display control unit causes the display device to display quality information of a classification corresponding to an operation specified based on information input using an input device. The computer
generating work data by attaching tag information related to the work at the time of acquisition of a microscope image acquired by the microscope device during an operation of assembling or inspecting a device using the microscope device;
generating quality information, which is statistical information related to the quality of the work, for each classification of the work data classified by the tag information based on the generated plurality of work data;
A work improvement support method, comprising the steps of: displaying the generated quality information on a display device. On the computer,
A process of generating work data by attaching tag information related to the work at the time of acquisition of a microscope image acquired by the microscope device during an operation of assembling or inspecting a device using the microscope device;
A process of generating quality information, which is statistical information related to the quality of the work, for each classification of the work data classified by the tag information based on the generated multiple work data, and a process of displaying the generated quality information on a display device;
A program characterized by executing the above.

Images