WO2021095519A1 - Information processing device - Google Patents

Abstract

According to the present invention, whether a learned model can demonstrate a capability obtained via machine learning is determined on the basis of data input into the learned model. An information processing device according to the present invention comprises an encoder, which includes an encoder model, and a determination unit. The determination unit receives specific feature amounts of the input data from the encoder model and outputs the degree of matching, to at least one data set, of the input data. The encoder model uses machine learning to perform matching so as to distribute feature amounts in a specific subspace of a feature amount space defined by a plurality of dimensions. The determination unit calculates the degree of matching of the input data to the data set from the positional relationship between the specific feature amounts and the specific subspace.

Description

Information processing device

This disclosure relates to an information processing device.

Conventionally, a configuration is known in which the feature amount input to the trained model is extracted from the input data. For example, Non-Patent Document 1 discloses a multi-task adversarial network (MTAN). In MTAN, feature representations of images labeled with content and style are generalized with respect to style factors through hostile learning in style identification of the image. According to MTAN, it is possible to generalize the feature representation of an image without causing confusion in the content identification of the image containing an unknown style factor.

Yang Liu, Zhaowen Wang, Hailin Jin and Ian Wassell, "Multi-Task Adversarial Network for Disentangled Feature Learning", in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2018, 375.pp.

Data acquired under conditions different from the conditions under which the training data was acquired can be input to the trained model. Even if the attributes of the data are the same, the feature amount of the data may differ greatly depending on the conditions under which the data is acquired. Depending on the degree of divergence between the two conditions, the performance of the trained model for the input data can be significantly reduced. However, in MTAN disclosed in Non-Patent Document 1, the performance deterioration of the trained model due to the discrepancy between the two conditions is not taken into consideration.

The present disclosure has been made to solve the above-mentioned problems, and the purpose of the present disclosure is to determine whether or not the trained model can exhibit the performance obtained by machine learning. The judgment is based on the input data.

According to an example of the present disclosure, the information processing device includes an encoder and a determination unit. The encoder includes an encoder model that outputs features of a plurality of dimensions from training data contained in at least one data set. The determination unit receives the specific feature amount of at least one input data from the encoder model, and outputs the goodness of fit of the at least one input data to the at least one data set. The encoder model is adapted by machine learning to distribute features in specific subspaces of the feature space defined by multiple dimensions. The determination unit calculates the goodness of fit of at least one input data to at least one data set from the positional relationship between the specific feature amount and the specific subspace.

According to this disclosure, it is possible to identify whether or not the input data is suitable for at least one environment (known environment) from which the training data is acquired by referring to the goodness of fit calculated by the determination unit. it can. As a result, it is possible to identify whether or not the trained model adapted by machine learning using the training data acquired in the known environment functions normally with respect to the input data. That is, it is possible to determine whether or not the trained model can exhibit the performance obtained by machine learning based on the input data to the trained model.

In the above disclosure, the determination unit back-calculates the difference between the specific feature amount and the feature amount extracted from the training data by the encoder model as the change amount of at least one input data via the encoder model, and the change amount. The method of changing the condition from which at least one input data for reducing the absolute value of is acquired may be output.

According to this disclosure, even if the input data does not conform to the known environment, the conditions of the environment in which the input data is acquired can be adapted to the conditions of the known environment. As a result, the trained model can exhibit the performance obtained in machine learning even for the input data acquired in the unknown environment.

In the above disclosure, the information processing device may further include a storage unit and a learning unit. At least one data set may be stored in the storage unit. The learning unit may adapt the encoder model so that the features are distributed in a specific subspace by machine learning using at least one data set.

According to this disclosure, an encoder model that has not been machine-learned can be adapted to an encoder model that has been learned by the learning unit.

According to an example of the present disclosure, the information processing device includes a storage unit, an encoder, and a learning unit. At least one data set is stored in the storage unit. The encoder includes an encoder model that extracts features of multiple dimensions from the training data contained in the dataset. The learning unit adapts the encoder model so that the features are distributed in a specific subspace of the feature space defined by a plurality of dimensions by machine learning using a data set.

According to this disclosure, an encoder model that has not been machine-learned can be adapted to an encoder model that has been learned by the learning unit.

In the above disclosure, the information processing device may further include a decoder. The decoder may include a decoder model that decodes the features from the encoder. The learning unit may adapt the encoder model and the decoder model so that the features follow a standard normal distribution by machine learning using at least one data set.

According to this disclosure, the decoder model is adapted to a variational auto-encoder (VAE) by machine learning, and the features extracted from the training data by the encoder model are uniform in a specific subspace. The encoder model is adapted to be distributed in. As a result, the difference in the position of the feature amount extracted from the input data in the feature amount space becomes clear depending on whether or not the input data fits at least one data set. As a result, it is possible to more accurately determine whether or not the trained model can exhibit the performance obtained by machine learning based on the input data to the trained model.

In the above disclosure, the information processing device may further include a classifier. The classifier may include a discriminative model that identifies which of the at least one environment the training data is contained in. Machine learning may be hostile learning performed between the discriminative model and the encoder model. In hostile learning, the discriminative model is optimized to maximize the probability that the discriminative model will succeed in identifying the correct environment from which the training data was acquired, and the probability that the discriminative model will fail in identifying the correct environment. The encoder model may be optimized for maximization.

According to this disclosure, hostile learning removes the bias peculiar to the environment in which the data contained in each of at least one data set is acquired from the features extracted from the training data by the encoder model. Since the features output from the encoder model emphasize the features common to known environments, it is determined to the trained model whether or not the trained model can exhibit the performance obtained by machine learning. It becomes easy to judge based on the input data of.

In the above disclosure, the specific subspace may be a spherical surface centered on the origin of the feature space.

According to this disclosure, by making a specific subspace spherical, the distance from the origin of the features of the data acquired in an environment compatible with the known environment through machine learning for the encoder and the incompatibility with the known environment. It is easy to make a clear difference in the distance from the origin of the feature amount of the data acquired in the environment. As a result, it is possible to more clearly determine whether or not the trained model can exhibit the performance obtained by machine learning based on the input data to the trained model.

According to the present disclosure, it is possible to identify an environment in which it is difficult for the trained model to exhibit the performance obtained by machine learning.

It is a block diagram which shows the structure of the information processing apparatus which concerns on embodiment. It is a block diagram for demonstrating the machine learning performed on the encoder model and the decoder model by the learning part of FIG. It is a block diagram for demonstrating the hostile learning performed by the learning part of FIG. It is a block diagram for demonstrating the machine learning performed on the main purpose model by the learning part of FIG. It is a block diagram which shows the flow of the inference processing by the trained encoder model and the trained main purpose model. The distribution of features output from the encoder model of FIG. 5 is shown when the inference processing performed by the main purpose model is a logistic regression of multiclass classification using a softmax function. It is a flowchart which shows the flow of the conformity determination process performed by the determination part of FIG. The distribution of features output from the encoder model of FIG. 5 is shown when the inference processing performed by the main object model is logistic regression of two-class classification using a sigmoid function. It is a flowchart which shows the flow of another example of the conformity determination process performed by the determination part of FIG. It is a block diagram which shows the structure of the information processing apparatus which concerns on modification 1 of embodiment. It is a block diagram which shows the structure of the information processing apparatus which concerns on modification 2 of embodiment. It is the schematic which shows an example of the whole structure of the appearance inspection system including the information processing apparatus of FIG. It is a schematic diagram for demonstrating the image example of the work acquired by the image pickup part of FIG. It is a schematic block diagram of the information processing apparatus of FIG.

Hereinafter, the embodiment will be described in detail with reference to the drawings. In principle, the same or corresponding parts in the drawings are designated by the same reference numerals and the description is not repeated.

<Application example>
FIG. 1 is a block diagram showing a configuration of an information processing device 100 according to an embodiment. As shown in FIG. 1, the information processing device 100 includes a storage unit Stg, a learning unit Ln, an encoder Enc, a decoder Dec, a discriminator Dsc, a main purpose processor Mpr, and a determination unit Jdg. .. Data sets E1 to En are stored in the storage unit Stg. The encoder Enc includes an encoder model Mc. The decoder Dec includes a decoder model Md. The classifier Dsc includes a discriminative model Me. The main purpose processor Mpr includes a main purpose model Mm. Each of the encoder model Mc, the decoder model Md, the discriminative model Me, and the main purpose model Mm includes a neural network. The decoder Dec may not be included in the information processing device 100.

The data sets E1 to En include training data dt1 to dtn. The training data dt1 to dtn are labeled with each of the data sets E1 to En, and include correct answer data corresponding to the output of the main purpose processor Mpr. The labels attached to each of the data sets E1 to En correspond to the environment in which the data included in each data set was acquired. The environment, also called a domain, is determined by the conditions under which the data is retrieved. The difference in the conditions can be a factor (bias) that makes the distribution of the features of the learning data of the same attribute different. For example, the condition includes, for example, the date and time when the learning data was acquired, the place where the learning data was acquired, the set value of the device from which the learning data was acquired, the model of the device, and the like. The "n" in the datasets En and dtn is a natural number of 2 or more.

The encoder model Mc extracts features of a plurality of dimensions from the training data included in the data set Ds. The decoder model Md receives a feature amount from the encoder model Mc and decodes the feature amount. The discriminative model Me receives the feature amount from the encoder model Mc and identifies the data set including the data from which the feature amount is extracted. The main object model Mm receives a feature amount from the encoder model Mc and makes an inference such as pattern recognition for the data from which the feature amount is extracted. Functions of the main purpose model Mm include, for example, image recognition, natural language processing, and voice recognition.

The learning unit Ln uses the encoder model Mc and the decoder to distribute the features in a specific subspace of the feature space defined by the dimension of the features output from the encoder model Mc by machine learning using the data set Ds. Fit the model Md to the variant self-encoder. A ring loss is used as a loss function in the optimization of the encoder model Mc, and the specific subspace can be made spherical by adapting the encoder model Mc and the decoder model Md to the self-encoder. The radius of the spherical surface may use a predetermined value, or may be learned in the process of optimizing the encoder model Mc. Cross entropy may be used as the loss function to make the specific subspace a hyperplane.

The learning unit Ln performs machine learning on the encoder model Mc and the decoder model Md as shown in FIG. The encoder model Mc receives the learning data dtz and outputs the feature amount fz to the decoder model Md. The decoder model Md decodes the feature amount fz into data dcz. The learning unit Ln minimizes the error Ls1 between the data dcz and the learning data dtz, backpropagates the feature fz so that it follows a standard normal distribution, and performs neurals included in each of the encoder model Mc and the decoder model Md. Update network weights and biases. By the machine learning, the encoder model Mc and the decoder model Md are adapted to the trained self-encoder.

The learning unit Ln performs hostile learning as shown in FIG. 3 between the encoder model Mc and the discriminative model Me. In FIG. 3, the probability Ps is the probability that the discriminative model Me succeeds in identifying the correct data set Ex from which the training data dtx has been acquired. The probability Pf is the probability that the discriminative model Me fails to identify the correct data set Ex.

As shown in FIG. 3, the learning unit Ln optimizes the discriminative model Me so that the probability Ps is maximized. That is, the learning unit Ln performs backpropagation so as to minimize the error between the correct answer data corresponding to the label of the correct answer data set Ex and the output of the discriminative model Me, and the weight of the neural network included in the discriminative model Me. And update the bias.

On the contrary, the learning unit Ln optimizes the encoder model Mc so that the probability Pf is maximized. That is, the learning unit Ln performs backpropagation so as to maximize the error between the correct answer data corresponding to the label of the correct answer data set Ex and the output of the discriminative model Me, and the weight of the neural network included in the encoder model Mc. And update the bias.

In hostile learning, the encoder model Mc is optimized to reduce the discriminative accuracy of the dataset containing the training data by the discriminative model Me, and the discriminative model Me is optimized to improve the discriminative accuracy of the dataset containing the training data. Optimized for. Through hostile learning, the environment-specific bias in which the data contained in each of the datasets E1 to En is acquired is removed from the features output from the encoder model Mc.

The learning unit Ln performs machine learning as shown in FIG. 4 on the main object model Mm using the data set Ds. The encoder model Mc receives the learning data dty and outputs the feature amount fy to the main object model Mm. The main object model Mm receives the feature quantity fy and outputs the inference result dcy. The learning unit Ln performs backpropagation so as to minimize the error Ls2 between the inference result dcy and the correct answer data day, and updates the weight and bias of the neural network included in the main object model Mm. By the machine learning, the main purpose model Mm becomes a trained model fitted to the datasets E1 to En.

FIG. 5 is a block diagram showing the flow of inference processing by the trained encoder model Mc and the trained main purpose model Mm. As shown in FIG. 5, the feature amount f_un of the data dt_un acquired in the data set E_un including the data acquired in the unknown environment is converted into the trained main object model Mm via the trained encoder model Mc. When input, the main purpose model Mm functions normally for the feature amount depending on the degree of deviation between the condition from which the datasets E1 to En are acquired and the condition from which the data contained in the dataset E_un is acquired. May not.

Therefore, in the information processing apparatus 100, the data dt_un is adapted to the data sets E1 to En from the positional relationship between the feature amount f_un and the specific subspace in the feature amount space with respect to the feature amount f_un derived from the data set E_un. Output the degree ad_un. By referring to the goodness of fit ad_un, it is possible to clearly identify whether or not the data dt_un conforms to the datasets E1 to En. According to the information processing apparatus 100, it can be determined based on the data dt_un whether or not the trained main object model Mm exhibits the performance obtained by machine learning. As a mode of output of the determination result, it may be output whether or not the trained main object model Mm can exhibit the performance for the data dtdt_un, and a plurality of data sets E_un included in the data set E_un. From the goodness of fit of the data, it may be output whether or not the trained main object model Mm can exhibit the performance for the environment determined by the condition in which the data included in the data set E_un is acquired. From the goodness of fit of the data dtdt_un, the reliability of the output result of the trained main purpose model Mm may be output.

FIG. 6 shows the distribution of the features output from the encoder model Mc of FIG. 5 when the inference processing performed by the main object model Mm is a logistic regression of multi-class classification using the softmax function. In FIG. 6, the feature space is drawn as a two-dimensional plane defined by the dimensions x1 and x2 for convenience of explanation, but the feature output from the encoder model Mc has three or more dimensions. In some cases. In FIG. 6, the features output from the encoder model Mc are plotted as points. The specific subspace Sb1 is drawn as a circle with a radius R1. The same applies to FIG. 8 which will be described later.

With reference to FIG. 6, the features of the data matching the datasets E1 to En are reflected in the distribution of the features extracted from the data, and the norm of the features (distance from the origin in the feature space) is It often has a certain size. Therefore, the feature quantities of the data acquired in the environment conforming to the datasets E1 to En are often distributed in the vicinity of the specific subspace Sb1. On the other hand, the features of the data acquired in the environment incompatible with the datasets E1 to En are often not reflected in the distribution of the features extracted from the data, and the norm of the features is often relatively small. .. Therefore, the feature quantities of the data acquired in the environment incompatible with the datasets E1 to En are distributed near the origin.

The distance δ between the feature amount f_un of the data dt_un and the specific subspace Sb1 represents the proximity of the data dt_un to the data sets E1 to En. Therefore, the determination unit Jdg outputs the goodness of fit ad_un of the data dt_un to the data sets E1 to En based on the distance δ. In FIG. 6, the distance δ is the distance between the point P_un corresponding to the feature amount f_un and the intersection Psb of the straight line passing through the point P_un and the origin and the specific subspace Sb1.

FIG. 7 is a flowchart showing the flow of conformity determination processing performed by the determination unit Jdg of FIG. In the following, the step is simply referred to as S. As shown in FIG. 7, the determination unit Jdg determines whether or not the distance δ is shorter than the threshold value δth in S11. The threshold value δth can be appropriately determined by an actual machine experiment or a simulation.

When the distance δ is smaller than the threshold value δth (YES in S11), the determination unit Jdg sets a value (for example, TRUE) indicating that the data dt_un conforms to the data sets E1 to En in S12 to the goodness of fit ad_un. And the process proceeds to S14. When the distance δ is equal to or greater than the threshold value δth (NO in S11), the determination unit Jdg sets a value (for example, FALSE) indicating that the data dt_un is incompatible with the datasets E1 to En in S13 in the goodness of fit ad_un. The process proceeds to S14.

The determination unit Jdg outputs the goodness of fit ad_un in S15 and ends the process. In FIG. 7, the case where the goodness of fit ad_un is set to either of the binary values indicating conformity or nonconformity has been described, but the goodness of fit ad_un is set to a continuous value (for example, a percentage) according to the distance δ. May be done.

If NO in S11, the determination unit Jdg may output a method of changing the condition in which the data dt_un for adapting the data dt_un to the data sets E1 to En is acquired. In this case, the determination unit Jdg inputs the difference between the feature amount f_un and the feature amount extracted by the encoder model Mc from the training data acquired in the data sets E1 to En into the encoder model Mc by backpropagation. Calculate back as the amount of data change. The determination unit Jdg outputs a method of changing the condition necessary for reducing the absolute value of the change amount. By executing the change method, the user can acquire data that fits the data sets E1 to En and that the trained main purpose model Mm can exhibit its performance.

FIG. 8 shows the distribution of the features output from the encoder model Mc of FIG. 5 when the inference processing performed by the main object model Mm is a logistic regression of two-class classification using a sigmoid function. With reference to FIG. 8, in the case of two-class classification, the sphere in which the features of the data conforming to the datasets E1 to En are distributed and the sphere in which the features of the data not conforming to the datasets E1 to En are distributed are Clear separation is possible in the optimization of the encoder model Mc using ring loss. In FIG. 8, the feature amounts of the data conforming to the datasets E1 to En are distributed near the spherical surface Sb1 having the radius R1, and the feature amounts of the data not conforming to the datasets E1 to En are distributed near the spherical surface Sb2 having the radius R2. ing. In such a case, the goodness of fit ad_un of the data dt_un to the datasets E1 to En can be calculated depending on whether the norm R of the feature amount f_un of the data dt_un is larger than the threshold value Rth larger than the radius R2. The threshold value Rth can be appropriately determined by an actual machine experiment or a simulation.

FIG. 9 is a flowchart showing the flow of another example of the conformity determination process performed by the determination unit Jdg of FIG. The flowchart shown in FIG. 9 is a flowchart in which S11 in FIG. 7 is replaced with S21.

As shown in FIG. 9, the determination unit Jdg determines in S21 whether or not the norm R is larger than the threshold value Rth. When the norm R is larger than the threshold value Rth (YES in S21), the determination unit Jdg determines that the data dt_un conforms to the data sets E1 to En, performs S12 and S14, and ends the process. When the norm R is equal to or less than the threshold value Rth (NO in S21), the determination unit Jdg determines that the data dt_un is incompatible with the data sets E1 to En, performs S13 and S14, and ends the process.

The conformity determination process by the determination unit Jdg is not limited to the rule-based determination method as shown in FIGS. 7 and 9, and may be a determination method using a trained model optimized by machine learning. .. In this case, the feature amount distributed in the specific subspace Sb1 and the invalid data in which noise is added to the feature amount can be used as learning data used for machine learning of the determination unit Jdg.

[Modification of the information processing device according to the embodiment]
In the information processing apparatus 100, a case where machine learning and inference by a trained model adapted by the machine learning are performed in the same apparatus has been described. Both may be done in separate devices.

FIG. 10 is a block diagram showing the configuration of the information processing device 100A according to the first modification of the embodiment. In the configuration of the information processing device 100A, the determination unit Jdg is removed from the information processing device 100 of FIG. 1, and the encoder Enc, the decoder Dec, the discriminator Dsc, and the main purpose processor Mpr are the encoder EncA, the decoder DecA, and the discriminating unit. It is a configuration replaced by the vessel DscA and the main purpose processor MprA. With reference to FIG. 10, the information processing apparatus 100A performs learning processing as shown in FIGS. 2 to 4 on the encoder model Mc, the decoder model Md, the identification model Me, and the main object model Mm.

FIG. 11 is a block diagram showing the configuration of the information processing device 100B according to the second modification of the embodiment. In the configuration of the information processing device 100B, the storage unit Stg, the decoder Dec, and the classifier Dsc are removed from the information processing device 100 of FIG. 1, and the encoder Enc and the main purpose processor Mpr are the encoder EncB and the main purpose processor. It is a configuration in which each is replaced with MprB. With reference to FIGS. 10 and 11, the encoder EncB and the main purpose processor MprB include an encoder model Mc and a main purpose model Mm adapted by the information processing apparatus 100A, respectively. In the information processing apparatus 100B, inference processing using the encoder model Mc and the main purpose model Mm, and conformity determination processing as shown in FIG. 7 or FIG. 9 are performed.

[Specific example of information processing device 100]
FIG. 12 is a schematic view showing an example of the overall configuration of the visual inspection system 1 including the information processing apparatus 100 of FIG. In the visual inspection system 1, the information processing device 100 functions as an image processing device. As shown in FIG. 12, the visual inspection system 1 detects defects in the work 2 based on an image obtained by incorporating an inspection target 2 (hereinafter, also referred to as “work 2”) into a production line or the like. Classify. That is, the main purpose processor Mpr of the information processing device 100 functions as an image classifier in the visual inspection system 1.

In the visual inspection system 1, the work 2 is conveyed in a predetermined direction by a conveying mechanism 6 such as a belt conveyor. On the other hand, the imaging unit 8 is arranged at a fixed position with respect to the work 2. Further, the illumination light source 9 is arranged at a fixed relative position with respect to the image pickup unit 8. The illumination light source 9 illuminates at least the field of view of the imaging unit 8 (the range in which the work 2 can be located). The imaging unit 8 images the moving work 2. The image data obtained by the imaging unit 8 is transmitted to the information processing device 100. The orientation of the image pickup unit 8, the amount of light of the illumination light source 9, the number of installations, and the arrangement position can be conditions for determining the environment in which the learning data is acquired. It is preferable that the amount of light, the number of installations, the arrangement position, etc. are optimized so as not to be disturbed by the surrounding lighting environment.

The fact that the work 2 has reached the field of view of the imaging unit 8 is detected by the detection sensors 4 arranged at both ends of the transport mechanism 6. Specifically, the detection sensor 4 includes a light receiving unit 4a and a light projecting unit 4b arranged on the same optical axis, and receives light that the light emitted from the light projecting unit 4b is shielded by the work 2. By detecting in the part 4a, the arrival of the work 2 is detected. The detection signal of the detection sensor 4 (hereinafter, also referred to as “trigger signal”) is output to the PLC (Programmable Logic Controller) 5.

The PLC 5 receives a trigger signal from the detection sensor 4 and the like, and controls the transport mechanism 6 itself.

The visual inspection system 1 further includes an information processing device 100, a display 102, and a mouse 104. The information processing device 100 is connected to the PLC 5, the imaging unit 8, the display 102, and the mouse 104.

As an example, the image pickup unit 8 includes an image sensor divided into a plurality of pixels, such as a CCD (Coupled Charged Device) and a CMOS (Complementary Metal Oxide Semiconductor) sensor, in addition to an optical system such as a lens.

The information processing device 100 is a computer having a general-purpose architecture. The information processing device 100 realizes various functions such as machine learning and defect classification by executing a pre-installed program. When using such a general-purpose computer, an OS (Operating System) for providing the basic functions of the computer is installed in addition to the application for providing the functions of the information processing device 100. You may. In this case, the program that realizes the function of the information processing apparatus 100 calls the necessary modules in a predetermined array at a predetermined timing among the program modules provided as a part of the OS to execute the process. There may be. That is, the program itself does not include the above-mentioned module, and the process is executed in cooperation with the OS. The program may be in a form that does not include such a part of modules.

Further, the program that realizes the function of the information processing apparatus 100 may be provided by being incorporated into a part of another program. Even in that case, the program itself does not include the modules included in the other programs to be combined as described above, and the processing is executed in cooperation with the other programs. That is, the program that realizes the function of the information processing device 100 may be in a form incorporated in such another program. Note that some or all of the functions provided by executing the program may be implemented as a dedicated hardware circuit.

FIG. 13 is a schematic diagram for explaining an image example of the work 2 acquired by the imaging unit 8 of FIG. FIG. 13A is an image of the work 2 having no defects. FIG. 13B is an image of the work 2 having the chipped D1. FIG. 13C is an image of the work 2 having the scratch D2. Defects that can occur in the work 2 are not limited to chips and scratches, but also include, for example, dents and deformations. The information processing apparatus 100 classifies the images of the work 2 according to the types of defects occurring in the work 2 (no defects, chips, scratches, dents, deformations, etc.).

FIG. 14 is a schematic configuration diagram of the information processing device 100 of FIG. As shown in FIG. 14, the information processing apparatus 100 includes a processor 110 as an arithmetic processing unit, a main memory 112 and a hard disk 114 as a storage unit, a camera interface 116, an input interface 118, a display controller 120, and the like. It includes a PLC interface 122, a communication interface 124, and a data reader / writer 126. Each of these parts is connected to each other via a bus 128 so as to be capable of data communication.

The processor 110 includes a CPU (Central Processing Unit). The processor 110 may further include a GPU (Graphics Processing Unit). The processor 110 expands the programs (codes) stored in the hard disk 114 into the main memory 112 and executes them in a predetermined order to perform various operations.

The main memory 112 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory). In addition to the program read from the hard disk 114, the main memory 112 holds image data acquired by the imaging unit 8, data indicating the processing result of the image data, work data, and the like.

The hard disk 114 is a non-volatile magnetic storage device. The hard disk 114 stores data sets E1 to En, a main purpose model Mm, an encoder model Mc, a decoder model Md, and an identification model Me, a machine learning program Pg1, and a defect identification program Pg2. In the visual inspection system 1, the main object model Mm functions as an image identification model. Various setting values and the like may be stored in the hard disk 114. The program installed on the hard disk 114 is distributed in a state of being stored in a memory card 106 or the like, as will be described later. In addition to the hard disk 114, or instead of the hard disk 114, a semiconductor storage device such as a flash memory may be adopted.

Each of the plurality of training data included in the data set Ds is an image of the work 2 labeled for each defect type. In addition, each of the plurality of learning data is also labeled according to the environment in which each learning data was acquired. The image of the work 2 may be an image taken by the image pickup unit 8 of FIG. 12 or an image taken by another image pickup device. The conditions under which the learning data is acquired include, for example, the orientation of the imaging unit 8, the amount of light of the illumination light source 9, the number of installations, and the arrangement position.

In the machine learning program Pg1, the data set Ds, the encoder model Mc, the decoder model Md, the discriminative model Me, and the main purpose model Mm are referred to. The processor 110 that executes the machine learning program Pg1 realizes the encoder Enc and decoder Dec of FIG. 2, the encoder Enc and discriminator Dsc of FIG. 3, and the main purpose processor Mpr of FIG. The processor 110 fits each of the encoder model Mc, the decoder model Md, the discriminative model Me, and the main purpose model Mm into the trained model by executing the machine learning program Pg1.

In the defect identification program Pg2, the encoder model Mc and the main purpose model Mm are referred to. The processor 110 that executes the defect identification program Pg2 realizes the encoder Enc of FIG. 5, the main purpose processor Mpr, and the determination unit Jdg. The processor 110 identifies defects in the image acquired by the imaging unit 8 by executing the defect identification program Pg2, and outputs the identification result to the display 102. The processor 110 outputs to the display 102 the goodness of fit of the environment in which the image is acquired to the data sets E1 to En. The goodness of fit is calculated by the determination unit Jdg according to the degree of similarity between the condition from which the training data is acquired and the condition from which the data input to the main purpose processor Mpr in the inference process is acquired. Depending on whether or not the goodness of fit exceeds a predetermined threshold value, the determination unit Jdg displays on the display 102 whether or not the input data is suitable for the environment determined by the conditions under which the learning data is acquired. It may be output.

When the input data to the main purpose processor Mpr does not conform to the data sets E1 to En, the processor 110 determines the conditions under which the input data is acquired in order to adapt the input data to the datasets E1 to En. The change method is output to the display 102. The change method includes, for example, changing the amount of light of the illumination light source 9, the number of installations, the arrangement, or the orientation of the imaging unit 8.

The camera interface 116 mediates data transmission between the processor 110 and the imaging unit 8. That is, the camera interface 116 connects the imaging unit 8 that images the work 2 and generates image data. More specifically, the camera interface 116 can be connected to one or more image pickup units 8 and includes an image buffer 116a for temporarily accumulating a plurality of image data from the image pickup unit 8. Then, when the image data for at least one frame is accumulated in the image buffer 116a, the camera interface 116 transfers the accumulated data to the main memory 112. Further, the camera interface 116 gives an imaging command to the imaging unit 8 according to an internal command generated by the CPU 110.

The input interface 118 mediates data transmission between the processor 110 and input units such as a mouse 104, a keyboard, and a touch panel. That is, the input interface 118 receives an operation command given by the user operating the input unit.

The display controller 120 is connected to a display 102, which is a typical example of a display device, and notifies the user of the result of image processing in the processor 110 and the like. That is, the display controller 120 is connected to the display 102 and controls the display on the display 102. The display 102 is, for example, a liquid crystal display, an organic EL (Electro Luminescence) display, or other display device.

The PLC interface 122 mediates data transmission between the processor 110 and the PLC 5. More specifically, the PLC interface 122 transmits information related to the state of the production line controlled by the PLC 5 and information related to the work to the processor 110.

The communication interface 124 mediates data transmission between the processor 110 and a console (or a personal computer or server device). The communication interface 124 typically comprises Ethernet (registered trademark), USB (Universal Serial Bus), or the like. As will be described later, instead of installing the program stored in the memory card 106 in the information processing device 100, the program downloaded from the distribution server or the like is installed in the information processing device 100 via the communication interface 124. You may.

The data reader / writer 126 mediates data transmission between the processor 110 and the memory card 106, which is a recording medium. That is, the memory card 106 is distributed in a state in which a program or the like executed by the information processing device 100 is stored, and the data reader / writer 126 reads the program from the memory card 106. Further, the data reader / writer 126 writes the image data acquired by the imaging unit 8 and / or the processing result in the information processing apparatus 100 to the memory card 106 in response to the internal command of the processor 110. The memory card 106 is a general-purpose semiconductor storage device such as CF (Compact Flash) or SD (Secure Digital), a magnetic storage medium such as a flexible disk (Flexible Disk), or a CD-ROM (Compact Disk Read Only Memory). ) And other optical storage media.

Further, another output device such as a printer may be connected to the information processing device 100, if necessary.

The system to which the information processing device according to the embodiment can be applied is not limited to the visual inspection system. Examples of the system include an automatic driving system that detects objects such as pedestrians and vehicles, and a medical diagnosis system.

In the automatic driving system, for example, the presence or absence of an obstacle such as a person or another automobile is identified in the image data taken from the driver's seat of the automobile. As a condition for determining the environment in which the image data is acquired, the time zone, the weather, the season, etc. in which the image was acquired can be mentioned. In the automatic driving system, it is determined in real time whether or not it is possible to determine the presence or absence of an obstacle in the image captured from the driver's seat during actual automatic driving.

In the medical diagnosis system, for example, daily photographs of patients, X-rays, and CT (Computed Tomography) are input to identify the illness and physical condition of the patient. Conditions that determine the environment in which the data was obtained include, for example, the device from which the data was obtained, the patient's age, gender, physique, constitution, nationality, and medical history. At the time of actual diagnosis by the medical diagnosis system, it is determined whether or not the diagnosis for the patient is possible.

Other systems to which the information processing device according to the embodiment can be applied include a face recognition system, a voice recognition system, a natural language processing system, a traffic condition monitoring system for predicting congestion, and a robot system for enhanced learning. Can be mentioned. In the face recognition system, a person is identified by a face image, and the device that acquired the face image, the age, gender, nationality, etc. of the person are included in the conditions for determining the environment in which the face image is acquired. .. In a voice recognition system, a person is identified by voice, and the device that acquired the voice, the age, gender, nationality, etc. of the person who made the voice are included in the conditions for determining the environment in which the voice was acquired. Is done. In a natural language processing system, the meaning of a character string is identified, and the environment is the language from which the character string is derived, the age, gender, nationality of the person who described the character string, and the field of the content that the character string means. Is included in the conditions for determining. In the traffic condition monitoring system, the presence or absence of traffic congestion is identified based on the road image, and the location of the road, the season when the road image was acquired, the time zone, the weather, the road surface condition, etc. determine the environment in which the road image was acquired. It is included in the conditions to be used. In a robot system, actions are identified based on image and voice recognition, and the location where the robot is placed, the performance and type of sensors provided by the robot, etc. determine the environment in which the image and sound are acquired. include.

As described above, according to the information processing apparatus according to the embodiment, it is possible to identify an environment in which it is difficult for the trained model to exhibit the performance obtained by machine learning.

<Additional notes>
The above-described embodiments include the following technical ideas.

(Structure 1)
An encoder (Enc) including an encoder model (Mc) that outputs features (fx) of a plurality of dimensions (x1, x2) from training data (dtx) included in at least one data set (E1 to En).
A specific feature amount (f_un) of at least one input data (dt_un) is received from the encoder model (Mc), and the goodness of fit (ad_un) of the at least one input data to the at least one data set (E1 to En). ) Is provided with a judgment unit (Jdg) that outputs
The encoder model (Mc) is adapted by machine learning so that the feature amount (fx) is distributed in a specific subspace (Sb1) of the feature amount space defined by the plurality of dimensions (x1, x2). ,
The determination unit (Jdg) is an information processing device (100, 100B) that calculates the goodness of fit (ad_un) from the positional relationship (δ) between the specific feature amount (f_un) and the specific subspace (Sb1).

(Structure 2)
The determination unit (Jdg) determines the difference between the specific feature amount (f_un) and the feature amount (fx) extracted from the training data (dtx) by the encoder model (Me). Is calculated back as the change amount of the at least one input data (dt_un), and the method of changing the condition in which the at least one input data (dt_un) is acquired for reducing the absolute value of the change amount is output. , The information processing apparatus (100, 100B) according to the configuration 1.

(Structure 3)
A storage unit (Stg, 114) in which at least one (E1 to En) is stored, and
By machine learning using at least one (E1 to En), a learning unit (Ln) that adapts the encoder model (Mc) so as to distribute the feature amount (fx) in the specific subspace (Sb1) is provided. The information processing apparatus (100) according to configuration 1 or 2, further comprising.

(Structure 4)
A storage unit (Stg) in which at least one data set (E1 to En) is stored, and
An encoder (Enc) including an encoder model (Mc) that extracts features (fx) of a plurality of dimensions (x1, x2) from training data (dtx) included in at least one data set (E1 to En).
Equipped with a learning department (Ln)
The learning unit (Ln) is subjected to machine learning using the at least one data set (E1 to En) to the specific subspace (Sb1) of the feature space defined by the plurality of dimensions (x1, x2). An information processing device (100A) that adapts the encoder model (Mc) so as to distribute a feature amount (fx).

(Structure 5)
A decoder (Dec) including a decoder model (Md) for decoding the feature amount (fz) is further provided.
The learning unit (Ln) uses the encoder model (Mc) and the decoder model (Mc) so that the feature quantity (fz) follows a standard normal distribution by machine learning using the at least one data set (E1 to En). The information processing apparatus (100, 100A) according to configuration 3 or 4, which is adapted to Md).

(Structure 6)
A classifier (Dsc) including a discriminative model (Me) for discriminating which of the at least one data set (E1 to En) the training data (dty) is included in is further provided.
The machine learning is hostile learning performed between the discriminative model (Me) and the encoder model (Mc).
In the hostile learning, the discriminative model (Me) is optimal so as to maximize the probability Ps that the discriminative model (Me) succeeds in discriminating the correct data set (Ey) including the learning data (dty). The information processing according to the configuration 5, wherein the encoder model (Mc) is optimized so as to maximize the probability that the discriminative model (Me) fails to identify the correct data set (Ey). Equipment (100,100A).

(Structure 7)
The information processing apparatus (100, 100A, 100B) according to any one of configurations 1 to 6, wherein the specific subspace (Sb1) is a spherical surface centered on the origin of the feature amount space.

The embodiments disclosed this time should be considered to be exemplary in all respects and not restrictive. The scope of the present invention is shown by the claims rather than the above description, and it is intended to include all modifications within the meaning and scope equivalent to the claims.

1 Visual inspection system, 2 Work, 4 Detection sensor, 4a Light receiving part, 4b Flooding part, 6 Conveying mechanism, 8 Imaging part, 9 Illumination light source, 14, Jdg judgment part, 100, 100A, 100B Information processing device, 102 display , 104 mouse, 106 memory card, 110 processor, 112 main memory, 114 hard disk, 116 camera interface, 116a image buffer, 118 input interface, 120 display controller, 122 interface, 124 communication interface, 126 writer, 128 bus, Ds dataset , Dec, DecA decoder, Dsc, DscA classifier, Enc, EncA, EncB encoder, Ln learning unit, Mc encoder model, Md decoder model, Me identification model, Mm main purpose model, Mpr, MprA, MprB main purpose processor.

Claims (7)

An encoder that includes an encoder model that outputs features of multiple dimensions from training data contained in at least one data set, and
It includes a determination unit that receives a specific feature amount of at least one input data from the encoder model and outputs the goodness of fit of the at least one input data to the at least one data set.
The encoder model is adapted by machine learning to distribute the features in a specific subspace of the features space defined by the plurality of dimensions.
The determination unit is an information processing device that calculates the goodness of fit from the positional relationship between the specific feature amount and the specific subspace. The determination unit back-calculates the difference between the specific feature amount and the feature amount extracted from the training data by the encoder model as a change amount of the at least one input data via the encoder model, and the change. The information processing apparatus according to claim 1, wherein the information processing apparatus according to claim 1 outputs a method of changing the condition from which at least one input data has been acquired in order to reduce the absolute value of the quantity. A storage unit in which at least one data set is stored, and a storage unit.
The information processing apparatus according to claim 1 or 2, further comprising a learning unit that adapts the encoder model so that the feature amount is distributed in the specific subspace by machine learning using the at least one data set. A storage unit in which at least one data set is stored, and
An encoder including an encoder model that extracts features of a plurality of dimensions from the training data contained in at least one data set, and an encoder.
Equipped with a learning department
The learning unit adapts the encoder model so as to distribute the feature amount in a specific subspace of the feature amount space defined by the plurality of dimensions by machine learning using the at least one data set. apparatus. A decoder including a decoder model that decodes the feature amount is further provided.
The information processing apparatus according to claim 3 or 4, wherein the learning unit adapts the encoder model and the decoder model so that the features follow a standard normal distribution by machine learning using the at least one data set. .. Further comprising a classifier including a discriminative model for discriminating which of the at least one dataset the training data is contained in.
The machine learning is hostile learning performed between the discriminative model and the encoder model.
In the hostile learning, the discriminative model is optimized so as to maximize the probability that the discriminative model will succeed in identifying the correct data set containing the training data, and the correct data set is identified. The information processing apparatus according to claim 5, wherein the encoder model is optimized so as to maximize the probability that the discriminative model will fail. The information processing device according to any one of claims 1 to 6, wherein the specific subspace is a spherical surface centered on the origin of the feature amount space.

Images