JP2006285489A - Associative memory and its software - Google Patents

Abstract

In an associative memory device using a conventional neural network, it is difficult to increase the storage capacity and to improve the performance of associative characteristics.
After initial learning in which initial learning data made up of a prototype corresponding to a classification category and its teacher signal is learned, the prototype's neighborhood test data made up of the prototype's neighborhood test input data and its teacher signal are used. The associative neural network in which the weighting coefficient obtained by repeating the additional learning of the prototype and the prototype neighborhood test data that became the incorrect answer output until the neighborhood test data of the prototype is correct is tested, A neural network for a ring memory in which a weighting factor obtained by learning learning data obtained by learning a ring inclination arrangement of a prototype that has been additionally learned and the prototype test input data that generated the incorrect answer output, and attractor convergence detection means, Attractor identification means and associative input identification Means, having a capital.
[Selection] Figure 1

Description

  The present invention relates to image recognition having a noise removal and restoration function for degraded images, various pattern recognitions, security systems such as unauthorized access prevention, or a mutual association using a multilayer neural network applicable to high-speed memory search. (Hetero-Associative) relates to a storage device.

  Types of associative memories using neural networks include self-associative memories and mutual (hetero) associative memories. Details are described in literature, Hopfield, JJ, Proc. Nationl. Acad. Sci. USA, 79, pp. 2554-2558, 1982, Associative Neural Memories Theory and Implementation Edited by Mohamad H. Hassoun OXFORD University Press 1993, etc. Yes.

  FIG. 3 shows an example of a configuration of a self-associative memory using one neural network as a conventional technique. A two-layer neural network or three-layer neural network consisting of only an input layer and an output layer (an hourglass neural network having one intermediate layer) or an interconnected network type neural network known as a hop field network is used. Yes. These are configured such that the output signal of the self-associative neural network is directly fed back to the input.

As a self-associative memory according to the prior art of FIG. 3, these operations will be briefly described in the case where a two-layer neural network (without an intermediate layer) of the self-associative neural network 33 is used.
First, under the control of the associative loop control processing unit 35, an associative input signal from the input terminal 11 is input to the self-associative neural network 33 via the input changeover switch 4 and the input terminal 12. The attractor convergence detection processing unit 34 is input and stored. When the output signal of the learned two-layer neural network as the self-associative neural network 33 is obtained at the output terminal 13, the self-associative neural network 33 receives the output signal via the input changeover switch 4 under the control of the associative loop control processing unit 35. Feedback is provided to the input terminal 12 of the associative neural network 33. For this reason, the number of input layer units is the same as the number of output layer units. Therefore, the learning input data (learning attractor) and the teacher signal are the same and are learned in advance. Here, in learning using a two-layer neural network, in many cases, the weight coefficient is obtained and set by calculation.

The associative loop control processing unit 35 feeds back the output signal from the self-associative neural network 33 to the self-associative neural network 33 through the input changeover switch 4 and the input terminal 12, and outputs it again from the self-associative neural network 33. Get a signal. This series of feedback processing is called associative loop processing, and the number of feedbacks is defined as the number of associative loops j.
Here, the input signal to the input layer of the self-associative neural network 33 at the input terminal 12 at the associative loop count j is X (j), and the output signal of the self-associative neural network 33 is X (j + 1). X (j + 1) is fed back to the input terminal 12 again in the associative loop process of the associative loop count j + 1.

When this associative loop process is repeated, the output signal from the previous associative loop process is the same and no longer changes, that is, the state of X (j) = X (j + 1) is the attractor convergence state. The output signal at that time is called an attractor, and is obtained from the output terminal 13 as an attractor output signal.
The attractor that can be converged may be a learning attractor that is learning input data having a feature closest to the associative input signal, or may be a spurious attractor (a false attractor). Spurious attractors are not learning input data, and in most cases are composed of data containing a lot of noise and do not have clear features.

  A series of operations from when the associative input signal X (0) at the input terminal 11 is input to when the attractor converges will be described below. First, the input switch 4 is initially set (when j = 0) by the associative loop control from the associative loop control processing unit 35, and the input terminal 11 and the input terminal 12 are connected via the input switch 4. Further, the internal state of the attractor convergence detection processing unit 34 is reset, and the associative input signal X (0) of the input terminal 11 is input and stored. Furthermore, the associative input signal X (0) is input as an input signal to the learned self-associative neural network 33 via the input changeover switch 4. An output signal X (1) is obtained from the self-associative neural network 33, and is input and stored in the attractor convergence detection processing unit 34 in order to detect whether or not the output signal X (1) has converged to attractors.

Further, the associative loop control processing unit 35 sets the number of associative loops j = 1, switches the input changeover switch 4 to feed back the output signal X (1) and inputs it to the self-associative neural network 33, and outputs the output signal X (1) is fed back to the self-associative neural network 33 via the input terminal 12 to obtain a new output signal X (2), which is input to the attractor convergence detection processing unit 34. Such an associative loop process is repeated.
Here, if X (0) is learning input data, that is, a learning attractor, X (0) and its output signal X (1) are equal to each other, and the attractor convergence detection state is immediately obtained. Is detected.

  That is, the attractor convergence detection processing unit 34 detects coincidence between X (j) and X (j + 1). However, j ≧ 0. If they match, it is assumed that the attractor has converged, and an attractor convergence signal is sent to the output terminal 20 and an associative loop process completion signal is sent to the associative loop control processing unit 35 so as to stop the associative loop process. The output signal of the self-associative neural network 33 at this time is sent to the output terminal 13 as an attractor output signal. The state converged on this attractor is called a recall state.

If they do not match, it is considered that they have not converged, and an associative loop processing request signal is sent out to request the associative loop control processing unit 35 again for associative loop processing.
In the associative loop control processing unit 35, when the associative loop processing request signal is received, if the associative loop processing number 35 is less than the pre-designated associative loop maximum number J, the associative loop number j = j + 1 is executed again. The output signal X (j + 1) is fed back to the input terminal 12 via the input changeover switch 4. On the other hand, when the associative loop process completion signal is received or when the associative loop count j reaches the maximum associative loop count J, the associative loop process is completed.

The associative loop processing is completed in the state where the number of associative loops j = J, and the attractor convergence detection processing unit 34 sends an attractor divergence signal to the output terminal 20.
In this state, even if the number of associative loops j is the specified maximum number J, the attractor convergence detection processing unit 34 cannot obtain the coincidence state of X (J) and X (J + 1). Means. This attractor divergence identification state is called a recollection state.

  The associative loop control processing unit 35 initializes the input selector switch 4 and the attractor convergence detection processing unit 34 every time a new associative input signal X (0) is input to the input terminal 11, and the series of processing described above. Is repeated for a new associative input signal X (0).

  The self-associative neural network 33 is a learned neural network in which weighting factors are set in advance. In the learning processing of the weighting factors, learning input data given as a learning attractor of the input terminal 11 and its output signal are used. The weighting coefficient is obtained by calculation and set based on the relationship that becomes the same learning attractor. The details of how to obtain this weighting factor are described in the above-mentioned document.

  When the associative input signal at the input terminal 11 is different from the learning attractor, an associative input signal similar to the learning attractor may reach the attractor convergence state through a plurality of associative loop processes and be recalled. For example, an unknown associative input signal including noise or the like may converge to a desired learning attractor by repeating the associative loop process, and the recall may be successful. Such an operation acts as a filter action or a content addressable memory.

  On the other hand, an attractor convergence signal is sent from the output terminal 20 of the attractor convergence detection processing unit 34, and at the same time, an attractor output signal different from the learning attractor is sent to the output terminal 13, and the attractor convergence called a spurious attractor (false attractor) is sent. The state may be recalled. In other words, for an associative input signal that contains a lot of noise and distortion, the attractor converges, but a spurious attractor that has almost no significant meaning that is completely different from a normal learning attractor is sent out as an attractor output signal. The In the application system using the attractor output signal converged on the spurious attractor, it causes a malfunction. However, the prior art does not perform attractor identification as a learning attractor or a spurious attractor for the attractor output signal.

  In order to make the correct recall in the conventional self-associative neural network 33 as successful as possible and suppress the occurrence of spurious attractors, the learning input data as learning attractors must have an orthogonal relationship that is not correlated with each other. That is, if a relatively similar associative input signal is calculated by calculating and setting a weighting factor for learning and storing as learning input data, interference between learning attractors occurs, and the case of convergence to spurious attractors increases, or There are more cases of non-convergent and non-convergent divergence.

  Therefore, the number of learning input data stored by the optimum weighting coefficient of the self-associative neural network 33, that is, the number of learning attractors is greatly limited. Further, when the number of learning input data increases, the neural network for self-association 33 has a low generalization capability, and attractor is converged only by a single feedback loop in the associative loop processing. However, there is a drawback in that the attractor tends to diverge, and even if it converges on the attractor, it tends to become a spurious attractor.

As described above, the biggest drawback of the self-associative memory using the self-associative neural network 33 is a small storage capacity due to a large restriction on a possible weight coefficient. According to the above document, this storage capacity is theoretically given by about 0.12 × number of input layer units.
Furthermore, when learning input data similar to each other is learned, there is a drawback that it is difficult to obtain an attractor convergence state from mutual interference. Further, in order to reduce spurious attractors in the attractor converged state that is erroneously recalled, the storage capacity is further reduced. In this way, attractor convergence becomes more difficult as the number of learned input data approaches the theoretical limit.

  On the other hand, as a self-associative neural network 33, instead of a two-layer neural network, a three-layer neural network in which a weighting coefficient obtained by adaptive updating of a weighting coefficient using a teacher signal in learning is set (one intermediate layer is Is used, the same operation as the above two-layer neural network is performed, and the generalization ability is relatively higher than that of the two-layer neural network, so that it is easy to converge on the attractor. The storage capacity is relatively large. The optimum number of intermediate layer units that maximizes the generalization rate is generally smaller than the number of units in the input layer and output layer, and is called an hourglass associative memory. Even if the number of intermediate layer units is increased, the associative characteristics cannot be improved.

The storage capacity is larger than in the case of the two-layer neural network, and if the number of learning attractors to be stored is relatively smaller than the number of input layer units, a high-performance associative characteristic can be realized. However, if the number of input layer units increases, the generalization ability deteriorates rapidly, so the associative ability declines, the recognition rate, the convergence speed to the learning attractor, and the convergence to the spurious attractor are further reduced. In addition, there is a drawback that attractor divergence increases rapidly.
In such a conventional system, unless the spurious attractor output signal is compared with all stored learning attractors, it cannot be distinguished whether it is a learning attractor or a spurious attractor, and requires a large amount of processing.

  As another prior art, there is a method called a mutual (hetero) associative memory using two cascaded neural networks. As shown in FIG. 4, this method includes a forward neural network 32 and a backward neural network 31 that are learned by updating weighting factors using teacher signals.

  As the output signal of the forward neural network 32, only one of the units is 1 and the other R-1 units are 0. The binary output signal format of the R unit, ie, One out of R (however, , R is the number of output layer output units, corresponding to the number of classification categories), and learning is performed using a coarse encoded code called. Since it is easy to associate the teacher signal corresponding to the learning input data of the neural network with the classification category, it is adopted as the output signal format of most multilayer neural networks and is learned.

  On the other hand, in the backward neural network 31, the teacher signal of the forward neural network 32 is learned as learning input data. Further, as a teacher signal of the backward neural network 31, an output of the same number of units as the forward associative input signal X (0) from the input terminal 11 is provided for feedback input to the forward neural network 32 via the input changeover switch 4. It is necessary to have a form, and it is set as learning input data of the forward neural network 32, that is, a learning attractor.

  That is, each learning is performed so that learning input data to the forward neural network 32 is reproduced as an output signal of the backward neural network 31. The weighting factors learned by updating the weighting factors in this way are set as the weighting factors for the forward neural network 32 and the backward neural network 31, respectively.

  Here, these operations will be briefly described. The input selector switch 4, the associative loop control processing unit 35, and the attractor convergence detection processing unit 34 in FIG. 4 have the same configurations as those in FIG. As in the case of FIG. 3, the input changeover switch 4 is controlled by the control signal from the associative loop control processing unit 35, and the forward associative input signal X (0) at the input terminal 11 has been learned. The connection is made so as to be input through the terminal 12. In addition, the forward associative input signal X (0) is initially set in the attractor convergence detection processing unit 34. In the forward neural network 32, an output signal Y (0) corresponding to the forward associative input signal X (0) is transmitted from the output terminal 13, and further input to the learned backward neural network 31 via the input terminal 17. The corresponding output signal X (2) is fed back to the input terminal 12 via the input changeover switch 4 as an associative loop process under the control of the associative loop control processing unit 35, and also to the attractor convergence detection processing unit 34. The input is memorized.

  In the attractor convergence detection processing unit 34, when the stored output signal X (j) of the backward neural network 31 coincides with the output signal X (j + 1) when the number of associative loops is j, the attractor is detected. The attractor convergence signal is transmitted from the output terminal 20 and the output signal of the backward neural network 31 is transmitted from the output terminal 18 as the attractor output signal. Even in the associative loop count j = J, an attractor divergence signal is transmitted from the output terminal 20 in an attractor divergence state that does not converge on the attractor.

  In the configuration of the mutual associative memory using the conventional three-layer neural network, the learning input data and the teacher signal as necessary R learning attractors are learned by the forward neural network 32, so that the storage capacity by the weight coefficient is increased. However, the generalization ability is higher than that of the self-associative neural network 33 using the two-layer neural network, and the associative loop processing through the forward neural network 32 and the backward neural network 31 is performed, so that convergence to the attractor is also possible. Fast and good recognition rate.

  However, if the learning input data number R as a learning attractor is very large, the forward neural network 32 using the one out of R coarse code output format requires the number of R output layer units, which are the same. It is necessary to prepare the backward neural network 31 having the number of input layer units, and the scale of the neural network becomes very large. Therefore, there is a drawback that the amount of calculation of the learning process and the execution process of the coarse code output type forward neural network 32 and the backward neural network 31 is enormous.

Also, in general, when the distance between learning attractors is close to each other, the generalization rate is relatively high, so the attractor convergence speed is also relatively fast, but the sparse output signal format consisting of R units of coarse code is Since it is used, the number of cases of convergence to a spurious attractor is extremely high.
In addition, in the case of an associative memory using a forward neural network in which learning attractors with small correlations that are separated from each other are used, the generalization ability cannot be increased, and excellent associative characteristics cannot be obtained. ing.

  In the above description, the coarse code output type forward neural network 32 that learns using a teacher signal has been described. However, a dense code output type three-layer neural network may be used as the forward neural network 32. However, compared with the performance of the forward neural network 32 using the coarse code output type three-layer neural network, spurious attractors are less likely to occur, but the generalization characteristics are slightly lower, and it is easy to converge to an erroneous learning attractor. Recognition rate is low. Furthermore, although the performance deteriorates, a self-organizing network may be used, but it has the disadvantages described above.

  As described above, since the self-associative memory using the two-layer neural network has the same number of input / output units, the weighting factor is limited in order to converge to the learning attractor that is the learning input data. There is a disadvantage that a lot of learning input data cannot be stored. For example, the theoretical maximum learning input data number that can be stored in a two-layer neural network is 0.12 × input layer unit number, and has a disadvantage that the storage amount of the weight coefficient is very small.

  In addition, in the hourglass self-associative memory using a three-layer neural network, when the storage capacity is increased, if the number of learning attractors exceeds the number of input layer units, the generalization ability deteriorates rapidly and is still rough. Because it is an output signal format, there are very many cases where it converges to the spurious attractor state, and for an associative input signal that is greatly degraded in noise, there may be an attractor divergence state that does not reach the attractor convergence state. It also has drawbacks such as very many.

  As described above, conventional self-associative memories and mutual associative memories have many drawbacks in practical use. In the case of a self-associative memory using two-layer and three-layer neural networks, the storage capacity of the weighting coefficient is small and many learning attractors cannot be stored. Also, since the associative loop process is a single feedback process, it does not converge to the attractor and tends to diverge the attractor, but even if it converges to the attractor, it is not a desired learning attractor but a spurious.・ There are drawbacks such as being easy to become an attractor.

  On the other hand, in the conventional associative memory using the three-layer neural network, the storage capacity can be greatly increased. However, when the forward neural network 32 using the coarse code output type three-layer neural network is used, many learning attractors are stored. In order to achieve this, the number of output layer units of the forward neural network 32 must be made very large, the scale of the forward and backward neural networks 32 and 31 becomes very large, and naturally the amount of computation becomes enormous. At the same time, it has the disadvantage that correct learning becomes difficult. Also, for the same reason, it has a drawback that it is very easy to converge on the spurious attractor.

Moreover, in the associative memory using the forward neural network based on the dense code output type three-layer neural network, the generalization ability is slightly inferior to that in the case of the coarse code type three-layer neural network, so that spurious attractors are hardly generated. However, the recognition rate is slightly inferior.
Furthermore, in a forward neural network that learns many learning attractors that are far from each other and have a small correlation, the generalization ability is low, so the mutual associative memory using this has the disadvantage that it cannot realize excellent associative ability. Yes.
Further, the conventional associative memory does not perform attractor identification, and the application system using the attractor output signal has a defect that causes malfunction.
As described above, the associative memory has various drawbacks.

  The object of the present invention is to solve the above-mentioned problems, and can have a very large storage capacity as compared with a self-associative memory and a mutual associative memory using a conventional neural network, and falls into a spurious attractor. In addition, the attractor divergence state is very small, the recognition rate is very high, and there is a high-performance associative characteristic. Moreover, classification category identification, attractor identification, and whether the associative input signal is learning input data. It is an object of the present invention to provide an associative memory device that is capable of identifying associative input as non-learning input data and reading out learning input data corresponding to a converged attractor output signal and is suitable for various applications.

  In order to solve the above-mentioned problem, as a first means, learning data arranged in a ring gradient composed of self-associative learned neural network means 1 and learning input data of the self-associative learned neural network means 1 is used. Learned neural network means 5 for ring memory, attractor convergence detecting means 2 for detecting an attractor convergence state for an associative input signal inputted to self-associative learned neural network means 1, and learned for ring memory The learning input data stored in the neural network means 5 is sequentially read out, and at least the spurious attractor learning is performed by comparison with the attractor output signal converged on the attractor output from the self-associative learned neural network means 1. Attractor Constituting the auto-associative memory device characterized by having at least a attractor identification means 6 for the attractor identification.

  As a second means, the interactive learning forward learned neural network means 25 and an output signal corresponding to the input signal from the forward learned neural network means 25 are fed back to the forward learned neural network means 25 and the attractor convergence state is obtained. A backward-learned neural network means 31 that learns learning data obtained by reversing the learning input data and the teacher signal, which are learning data of the forward-learned neural network means 25, and The learned neural network means 5 for ring memory that has learned the learning data in which the learning input data of the learned neural network means 25 is arranged in a ring gradient, and the associative input that is input to the forward learned neural network means 25 Attractor convergence detecting means 2 for detecting the attractor convergence state for the signal and the learning input data stored in the ring memory learned neural network means 5 are sequentially read out and output from at least the backward learned neural network means 31 A mutual associative memory device comprising at least an attractor identifying means 6 for identifying an attractor as a learning attractor or a spurious attractor of the attractor output signal by comparing with an attractor output signal converged on the attractor. Constitute.

  As a third means, the forward-learned neural network means 25 for associative association and an output signal for the input signal from the forward-learned neural network means 25 are fed back to the forward-learned neural network means 25 and the attractor convergence state A backward-learned neural network means 31 that learns learning data obtained by reversing the learning input data and the teacher signal as learning data of the forward-followed neural network means 25, The learned neural network means 5 for the ring memory that has learned the learning data in which the learning input data of the forward learned neural network means 25 is arranged in a ring gradient, and the associative input that is input to the advanced learned neural network means 25 The attracting state of the attractor for the signal is detected, and the learning signal of the attractor output signal converged to the attractor output from the backward learned neural network means 31 is further detected using the output signal of the forward learned neural network means 25. The attractor detection / identification means 26 for identifying the attractor of the attractor and the attractor output signal output from the backward learned neural network means 31 are sequentially read from the learned neural network means 5 for the ring memory. It has at least an attractor coincidence detecting means 29 for performing at least a coincidence comparison between the learning input data and the attractor output signal and controlling reading of the learning input data from the learned neural network means 5 for the ring memory. Constituting the mutual associative memory apparatus characterized.

  As a fourth means, in the associative memory device of the first, second and third means, at least a coincidence comparison between the learned input data from the learned neural network means 5 for the ring memory and the associative input signal is performed. And an associative storage device characterized by comprising associative input identifying means 9 for performing learning / non-learning input data identification of the associative input signal.

  As fifth means, in the associative memory 4 device of the first, second, third, and fourth means, learning input for sequentially reading out and storing the learning input data from the learned neural network means 5 for the ring memory An associative memory device having the data memory means 8 is configured.

  As a sixth means, a learned neural network process for ring memory that has learned learning data arranged in a ring slope composed of learning input data of the self-associative learned neural network process 130 and the learned neural network process 130 for self-associative learning. 190, an attractor convergence detection process 140 for detecting an attractor convergence state with respect to the associative input signal input to the self-associative learned neural network process 130, and the learning stored in the ring memory learning neural network process 190 The input data is sequentially read out, and at least whether it is a learning attractor or a spurious attractor by comparing with the attractor output signal converged to the attractor output from the learned neural network processing 130 for self-association. Constituting the auto-associative memory software is characterized by having at least a attractor identification process 260 to the Rakuta identified.

  As a seventh means, the forward-learned neural network process 430 for associative association and an output signal corresponding to the input signal from the forward-learned neural network process 430 are fed back to the forward-learned neural network 430 and the attractor convergence state is set. A backward-learned neural network process 610 that learns learning data obtained by reversing the learning input data and the teacher signal, which are learning data of the forward-learned neural network process 430, and that sends out an attractor output signal; The neural network processing 490 for the ring memory that has learned the learning data in which the learning input data of the already-executed neural network processing 430 is arranged in a ring slope, and the continuous network that is input to the neural network processing 430 that has been forward-learned. Attractor convergence detection processing 440 for detecting an attractor convergence state with respect to an input signal, and the learning input data stored in the learned neural network processing 490 for the ring memory are sequentially read, and at least the backward learned neural network neural network processing 610 At least an attractor identification process 560 for discriminating the attractor output signal from the learning attractor or the spurious attractor by comparing the attractor output signal converged on the attractor output from the attractor. Configure storage software.

As an eighth means, a forward-learned neural network process 730 for mutual association and an output signal corresponding to an input signal from the forward-learning neural network process 730 are fed back to the forward-learned neural network process 730 and the attractor convergence state A backward-learned neural network process 610 for learning the learning data obtained by reversing the learning input data and the teacher signal, which is the learning data of the forward-learned neural network process 730, In the learned neural network process 730, the learned input data of the ring memory arranged in a ring gradient is used to learn the learned neural network process 490 for ring memory, and the advanced learned neural network process 730. The attractor convergence state with respect to the input associative input signal is detected, and the output signal of the forward learned neural network process 730 is used to output the attractor output signal converged on the attractor output from the backward learned neural network process 610. Based on the attractor identification process 740 for discriminating between the learning attractor and the spurious attractor, and the attractor output signal sent from the backward learned neural network process 610, the ring memory learned neural network process 490 sequentially reads. Attractor output science that performs at least a coincidence detection between the learning input data and the attractor output signal, and controls reading of the learning input data from the learned neural network processing 490 for the ring memory Constituting the mutual associative memory software is characterized by having at least the input data coincides state determination processing 520.
As a ninth means, in the associative memory software of the sixth, seventh and eighth means, at least a coincidence comparison between the learning input data from the learned neural network processing 730 for the ring memory and the associative input signal is performed. Performing associative input identifying processing 590, 290 for performing learning / non-learning input data identification of the associative input signal.
As tenth means, in the associative memory software of the sixth, seventh, eighth, and ninth means, learning for sequentially reading out and storing the learning input data from the learned neural network processing 190, 490 for ring memory The associative memory software is characterized by having the input data storage processes 200 and 500.
As eleventh means, in the first to fifth associative learned neural network means 1 and 25, initial learning data prepared in advance comprising a prototype corresponding to a classification category and a teacher signal thereof is used as a neural network. After the initial learning, the prototype neighborhood test input data consisting of the prototype neighborhood test data and its teacher signal is used to perform the test, and the prototype neighborhood test input data that gives an incorrect answer is added. Additional learning is performed with the prototype as learning input data, and the proximity test data additional learning process of the prototype is repeated for all of the prepared prototype neighborhood test input data until the correct answer is output, and the obtained weighting factor is set. Learned neural network means 1 and 25 It constitutes an associative memory device characterized by that there.
As a twelfth means, as the sixth to tenth associative learned neural network processes 130 and 430, the initial learning data prepared from the prototype corresponding to the classification category and its teacher signal is learned in the neural network. After the initial learning, the prototype neighborhood test input data consisting of the prototype neighborhood test data and its teacher signal is used to test the prototype neighborhood test input data, which is an incorrect answer output. Additional learning is performed with the prototype as the data, and the prototype neighborhood test data additional learning process is repeated until all of the prepared prototype neighborhood test input data are correctly output, and the associated weighting coefficient is set. Neural network processing 13 , Constituting the associative memory software characterized by using a 430,730.
In the initial learning, the associative memory device of the present invention is added by using the prototype neighborhood test data as additional learning input data in addition to the representative prototype as learning input data assigned to each classification category. The learned multi-layer neural network having a very high generalization ability is introduced as a self-associative memory self-associative memory network and a cross-associative memory forward neural network, respectively. For this reason, large-scale storage capacity can be realized, but the attractive force to the attractor state is strong, the attractor divergence state is greatly improved, the convergence to the spurious attractor is reduced, and the recognition rate is very high And can achieve excellent associative characteristics. Furthermore, by introducing a ring memory that stores additional learning input data in addition to the prototype corresponding to the classification category, a series of learning input data including the additional learning input data corresponding to the attractor identification and converged learning attractor is called. At the same time, learning / non-learning input data identification of the associative input signal can be performed, and it has an excellent feature not found in the conventional associative memory device.

  FIG. 1 shows a configuration example of a self-associative memory device having a ring memory and using an additional learning neural network in the first embodiment of the present invention. In the following self-associative memory device of the present invention, in order to simplify the description of the basic configuration, a three-layer neural network using binary input / output signals is taken as an example, and the configuration and operation thereof are described in detail. explain. The present invention is not limited to a three-layer neural network, and any neural network that learns using a teacher signal, such as a multilayer neural network having four or more layers, may be used.

  The self-associative memory device of FIG. 1 selects either an associative input signal input to the input terminal 11 or an output signal from the self-associative additional learning neural network 1 by a control signal from the associative loop control processing unit 3. , An input changeover switch 4 that is input to the self-associative additional learning neural network 1 and the attractor convergence detection processing unit 2 via the input terminal 12, an output signal is obtained for the input signal from the input terminal 12, and the attractor output signal And the self-association additional learning neural network 1 to be sent to the input changeover switch 4 and the attractor convergence detection processing unit 2 and the self-association addition based on the control from the associative loop control processing unit 3. The input signal and the output signal to the learning neural network 1 are respectively input and the attractor converges. The attractor convergence detection processing unit 2 for transmitting a signal or attractor divergence signal from the terminal 20 and the attractor output signal obtained via the output terminal 13 or the output signal of the ring memory neural network 5 are associated with the associative loop control processing unit 3. An input changeover switch 7 that switches to the ring memory neural network 5 via the input terminal 15 and an output signal corresponding to the input signal that is input via the input terminal 15. 6, learning input data memory 8, associative input identification processing unit 9, and ring memory neural network 5 to be sent to input selector switch 7 for feedback, attractor output signal from terminal 13 and ring memory neural network 5 Associative loop with learning input data as output signal A learning input data memory 8 that is stored under the control of the control processing unit 3 and that is output from the output terminal 16, an attractor output signal from the output terminal 13 and a neural network 5 for ring memory under the control of the associative loop control processing unit 3 Are respectively input, the attractor output signal is identified, the attractor identification signal is sent to the associative input identification processing unit 9 and the output terminal 14, and the control signal and attractor from the associative loop control processing unit 3. Based on the attractor identification signal from the identification processing unit 6, the learning / non-learning input data of the associative input signal is identified based on the output signal of the ring memory neural network 5 and the associative input signal X (0) from the input terminal 11. An associative input identification processing unit 9 for transmitting a learning / non-learning input data identification signal from the output terminal 21; an input selector switch 4; 7, attractor convergence detection processing section 2, associative input identification processing unit 9 and the attractor identification processor 6, and the learning input data memory 8 from the associative loop control processing unit 3 for each control.

  Hereinafter, a series of associative loop processing operations will be described. Prior to the associative loop processing, the additional learning neural network 1 for self-association sets the weighting factor obtained by executing the following neighborhood test data additional learning processing of the prototype in order to have high-performance generalization ability. The case will be described. In this learning process, first, a representative prototype P (r) assigned to each classification category is composed of learning input data indicating a learning attractor and a teacher signal corresponding to the classification category r. Initial learning is performed on the initial learning data to converge the three-layer neural network so that all correct answer outputs are obtained. Of course, a weighting coefficient only for initial learning in which only a prototype is learned may be set. Here, one prototype P (r) is prepared for each classification category. However, when there are a plurality of prototypes P (r), it is sufficient to handle data other than one as neighborhood test input data.

  Subsequently, using the prototype neighborhood test data consisting of the prototype neighborhood test input data prepared in advance and the teacher signal corresponding to the classification category, the initially learned three-layer neural network is tested. Neighboring test input data sending correct output is added as learning input data Z (r, n) (where n is the learning data number of classification category r and 0 ≦ n ≦ nr) along with initial learning input data The learning is performed, and additional learning is performed so that all correct output is transmitted and converged with respect to the learning input data. Thereafter, the same test is performed again. If there is an incorrect answer output, the neighboring test input data is added as additional learning input data, and the additional learning is performed again. Such additional learning processing is repeated, and when all the correct answers are obtained for the prototype neighborhood test input data, the additional learning processing is completed.

  When the prototype neighborhood test data additional learning process is completed, the weighting coefficient is set in the self-associative neural network and used as the self-associative additional learning neural network 1. The details of the additional learning process of the above three-layer neural network are disclosed in Japanese Patent Laid-Open No. 10-49509.

On the other hand, in the neural network 5 for ring memory, initial learning input data and learning input data Z (r, n) consisting of the prototype P (r) used in the above additional learning process (where 0 ≦ n ≦ nr and nr are the number of additional learning input data of the classification category r), and prepare ring memory learning data for the ring memory neural network 5 shown in Table 1, and for these learning input data, Learning is performed so that all the correct answer outputs are converged, and the obtained weight coefficient is set in the ring memory neural network 5 and used. In Table 1, the number of classification categories corresponding to the prototype P (r), that is, the number of prototypes is R, and the maximum value of the number nr of additional learning input data for each category is Nmax.

  In Table 1, the prototype P (r), which is the initial learning input data assigned to the classification category r, and the corresponding learning input data Z (r, n), 0 ≦ n ≦ nr are classified category groups. The relationship of learning data is shown. Since the prototype P (r) is assigned as the teacher signal corresponding to the last additional learning input data in the group, the learning input data arranged in the ring state can be stored. Here, the arrangement relationship between the learning input data and the teacher signal is referred to as a ring inclination arrangement. That is, learning by classification category consisting of learning input data consisting of prototype P (r) and nr pieces of learning input data Z (r, n), 0 ≦ n ≦ nr, and teacher signals in which these are ring-tilted Get the data.

  All the learning data for each classification category is learned, and the learning input data is converged so that all the correct answers are output. The weighting coefficient learned in this way is set in the ring memory neural network 5. Therefore, when the prototype P (r) is inputted to the ring memory neural network 5 via the terminal 15, the learning input data Z (r (r) is obtained by a series of output feedback read processing of the ring memory via the input changeover switch 7. , n) are read sequentially, and finally, the prototype P (r) is read. The number of all learning input data read loops in the group of the classification category r is given by nr + 1. Therefore, when the ring memory input signal that is a prototype is a prototype, when the same ring memory read output signal is obtained, a series of learning input data in the group is all read.

  Next, the operation of the self-associative memory device having the self-associative neural network 1 and the ring memory neural network 5 of the present embodiment will be described. The associative loop control processing unit 3 initializes the number of associative loops j = 0 and the ring memory read counter n = 0, and sets the input selector switch 4 and attractor convergence detection processing based on the associative loop control signal. Initial setting of the internal state of the unit 2 and the learning input data memory 8, the attractor identification processing unit 6 and the associative input identification processing unit 9 is performed. The associative input signal X (0) is input to the input changeover switch 4 and input to the self-associative additional learning neural network 1 via the input terminal 12, and is also input and stored in the attractor convergence detection processing unit 2.

The associative input signal X (0) is input and stored in the associative input identification processing unit 9. The additional learning neural network 1 for self-association sends an output signal X (1) corresponding to the input X (0) and inputs it to the attractor convergence detection processing unit 2. The attractor convergence detection processing unit 2 sets the associative loop number counter (j = 0) based on the control signal from the associative loop control processing unit 3 and compares the input X (0) and X (1). If they are different, the attractor non-convergence state is set, the associative loop count j is set to j = j + 1, and an associative loop processing request is sent to the associative loop control processing unit 3.
When there is an associative loop processing request, the associative loop control processing unit 3 inputs it to the self-associative additional learning neural network 1 and the attractor convergence detection processing unit 2 via the input changeover switch 4, and outputs the output signal X (2). obtain.

  In this way, a series of associative loop processing for detecting the attractor convergence state is repeated, and the attractor convergence detection processing unit 2 inputs the input signal X of the self-associative additional learning neural network 1 for the associative loop processing number j. When (j) and the output signal X (j + 1) match, it is considered that the signal has converged to the attractor, and an attractor convergence signal is sent to the associative loop control processing unit 3 and the attractor convergence detection process is terminated. Further, the attractor convergence signal is output to the output terminal 20. At this time, if the number of associative loops is j = 0, an attractor convergence signal (One-Shot attractor convergence signal) is transmitted from the output terminal 20. Further, if the inconsistency continues and the convergence of the attractor is not obtained even when the specified associative loop count J is reached, the attractor is a divergent attractor, and the attractor divergence signal is transmitted from the output terminal 20 to request the associative loop processing stop. And attractor convergence detection processing is terminated. Further, the associative loop control processing unit 3 sends an attractor divergence identification signal from the output terminal 14 of the attractor identification processing unit 6, sends a non-learning input identification signal from the terminal 21 of the associative input identification processing unit 21, and Complete the associative loop process.

  When the attractor convergence detection processing unit 2 detects the attractor convergence state and the attractor convergence signal is sent to the associative loop control processing unit 3, the self-associative additional learning neural network is controlled under the control of the associative loop control processing unit 3. The output signal of the network 1 is sent as an attractor output signal from the output terminal 18 to the ring memory neural network 5 via the input changeover switch 7 and the input terminal 15, and the attractor identification processing unit 6, the associative input identification processing unit 9 and learning In the input data memory 8, a series of associative loop processes including the following learning input data reading process, attractor identifying process and associative input identifying process are performed. Hereinafter, a series of these processes will be described.

  Under the control of the associative loop control processing unit 3, the attractor output signal X (j + 1) of the output terminal 13 of the self-associative additional learning neural network 1 is ringed via the input terminal 15 via the input changeover switch 7. The data is input to the memory neural network 5 and is also stored in the attractor identification processing unit 6. For the attractor output signal X (j + 1) from the input terminal 15, the output signal Rout (1) is obtained from the ring memory neural network 5, and the attractor identification processing unit 6, the associative input identification processing unit 9 and the learning input data are obtained. Each is input to the memory 8. Attractor identification processing unit 6 compares attractor output signal X (j + 1) and output Rout (1), and if they do not match, set ring memory read counter n = n + 1 (however, initial value n = 0) ), The input changeover switch 7 is controlled based on the control of the associative loop control processing unit 3, and Rout (1) is fed back to the ring memory neural network 5 through the input terminal 15 and the readout process is performed again.

The attractor output signal X (j + 1) and output Rout (n) match within the range where the ring memory read counter n (≧ 0) in the attractor identification processing unit 6 is the maximum number of read counters by category specified in advance. Until then, a series of processes by the above feedback is repeated. Note that the maximum number of read counters for each category is set to the number nr of additional learning input data for each category shown in Table 1, where the maximum value Nmax within (n1, n2,... Nr,... NR) is set. When X (j) and Rout (n) match as shown in Table 2, there is a learning data number nr having the same value as the ring memory read counter n at this time, and the corresponding prototype P (r ) And the attractor output signal X (j + 1) can be identified and identified as a learning attractor, and the learned attractor identification signal is sent to the associative input identification processing unit 9, the associative loop control processing unit 3, and the output terminal 14. Output each one.
Here, attractor identification conditions in the attractor identification processing unit 6 are shown together in Table 2.

  On the other hand, as shown in Table 2, the prototype P (r) of the classification category r having the same learning data number nr is compared with the attractor output signal X (j + 1). If there is no prototype having the number of learning data corresponding to the counter n, or there is no category number r of the learning data number nr that matches the ring memory read counter n with n ≦ Nmax even if they match, The attractor output signal X (j + 1) is found to be a spurious attractor, and a spurious attractor identification signal is transmitted. In addition, as shown in Table 2, the attractor output signal X (j) is also obtained when the attractor output signal X (j + 1) and the outputs Rout (n) and P (r) do not all match when the maximum number of readouts by category is Nmax or less. +1) is found to be a spurious attractor, and sends a spurious attractor identification signal to the associative input identification processing unit 9, the associative loop control processing unit 3 and the output terminal 14, respectively. In this manner, by using the ring memory read output signal, it is possible to identify the attractor output signal as a learning attractor or a spurious attractor.

When the attractor identification processing unit 6 completes the attractor identification processing, the associative loop control processing unit 3 reads the learning input data memory 8 in the case of learning attractor identification, and the prototype corresponding to the learning attractor from the output terminal 16. And all the additional learning input data.
The associative input identification processing unit 9 compares the associative input signal X (0) and Rout (n), and once they match, the associative input signal is the learning input data, so the learning input data match state is stored. To do. Here, when a learning attractor identification signal is input from the attractor identification processing unit 6, a learning input data identification signal is transmitted from the output terminal 21. When the learning attractor identification signal is input, if all the ring memory read output signals do not match the associative input signal X (0) and the learning input data mismatch state is stored, the non-learning input data identification signal is transmitted. Is done.

On the other hand, when the spurious / attractor identification signal is identified in the attractor identification processing unit 6, the spurious / attractor identification signal is transmitted from the output terminal 14, and learning is performed based on the control of the associative loop control processing unit 3. The internal state of the input data memory 8 is reset, the associative input identification processing unit 9 identifies the associative input signal X (0) as non-learning input data, sends out a non-learning input data identification signal from the output terminal 21, and all Complete the process.
As an initial setting, based on Table 1, a category-specific maximum read count number Nmax, a category-specific additional learning input data number nr, and a prototype P (r) thereof are stored in advance.

  As described above, when a prototype P (r) for each classification category, which is a learning attractor, is transmitted as an attractor output signal from the output terminal 13, the prototype P (r) and its neighboring additional learning input data are stored. With the ring memory readout counter n equal to the number nr of additional learning input data by category, X (j + 1) = Rout (nr) = P (r), and again by prototype, including the prototype (read out last) All learning input data can be read out.

When an attractor output signal, which is a spurious attractor, is transmitted from the output terminal 13, a spurious attractor identification signal is transmitted from the output terminal 14.
In the attractor identification processing unit 6, the classification category number r of the matched prototype P (r) may be transmitted from the output terminal 14 together with the learning attractor identification signal. Thereby, category classification can also be performed.

  Here, in the attractor identification processing unit 6, the attractor output signal X (j + 1) from the output terminal 13 is directly compared with the pre-stored prototype P (r), 1 ≦ r ≦ R, and either matches. Then, it may be identified as a learning attractor, and a learning attractor identification signal may be sent to the associative input identification processing unit 9, the associative loop control processing unit 3, and the output terminal 14.

As described above, with respect to the associative input signal X (0), in the associative loop processing, the attractor convergence state is detected as shown in Table 3, and if it converges on the attractor, the attractor output signal X ( j + 1), a series of learning input data is read out from the ring memory neural network 5, and as shown in Table 2, attractor identification as a learning attractor or a spurious attractor and learning / non-association of associative input signals are performed. Learning input can be identified.

Here, regarding the self-associative memory processing shown in FIG. 1 in the first embodiment of the present invention, the associative processing flow when the associative input signal X (0) is given is shown in FIG. Here, in the self-associative additional learning neural network processing 130 and the ring memory neural network processing 190, the processing flow will be described on the assumption that additional learning and learning have already been completed in the same manner as described above. .
The associative initial setting process 100 includes the maximum associative loop number J in the associative loop number determination process 160, the maximum read count number Nmax by category in the ring memory read number determination process 230, that is, the maximum in the classification category in Table 1. Initial parameter settings such as the maximum number of additional learning data in category Nmax = Max {n1, n2, .., nR} and the attractor detection and identification condition setting shown in Table 2 for attractor identification processing 260 Do each.

The associative loop initial setting process 110 is an initial setting of the association loop count j as an initial setting of j = 0 and ring memory read counter n = 0, associative input-ring memory output coincidence detection process 210 and learning input data storage process 200. Reset the state.
The associative input signal input setting process 120 sets the associative input signal X (0) to the self-associative additional learning neural network process 130, the attractor convergence detection process 140, and the associative input-ring memory output coincidence detection process 210, respectively.

  The additional learning neural network processing 130 for self-association is the case where the number of associative loops j = 0 and the binary output signal X (1) with respect to the input associative input signal X (0) or the number of associative loops j> 0 Is inputted with the binary output signal X (j) at the association loop count j-1 input and set through the self-associative additional learning neural network output signal input setting processing 180, and the binary output signal X ( Send j + 1) respectively.

The attractor convergence detection processing 140 is performed when the associative loop count j = 0, the output signal X (1) for the associative input signal X (0) of the self-associative incremental learning neural network processing 130, or the associative loop count j. , A match comparison between the binary input signal X (j) and the output signal X (j + 1) is performed to detect the attractor convergence state. If the input signal X (j) and the output signal X (j + 1) match, the attractor converges with the input / output signal serving as the attractor. At this time, if the number of associative loops j = 0, the One-Shot attractor converges, and if the number of associative loops j ≧ 1, the generalized attractor converges. If they do not match, the attractor is not converged.
When the attractor convergence state is obtained in the attractor convergence determination processing 150, a ring memory neural network processing 190 is performed. On the other hand, if the attractor has not converged, the associative loop number determination process 160 is continuously performed.

In the associative loop count determination process 160, if the associative loop count j is less than the maximum associative loop count J, the associative loop count j is incremented by 1 in the j ← j + 1 setting process 170.
In the self-associative additional learning neural network output signal input setting process 180, X (j + 1) is set as an input of the self-associative additional learning neural network process 130. Thereafter, the self-associative additional learning neural network processing 130, attractor convergence detection processing 140, and attractor convergence determination processing 150 are performed again, as in the case of the previous association loop count j.

  On the other hand, in the associative loop number determination process 160, if the associative loop number j = J, the associative input signal X (0) does not converge to the attractor in the associative loop process of the designated maximum number J, and the attractor It is determined that it has become a recollection in the divergent state. According to this determination result, in the associative input identification setting process 310, the associative input signal X (0) is identified as non-learning input data and is a divergent attractor, and all associative loop processes are completed.

When it is determined in the attractor convergence determination processing 150 that the attractor is in a converged state, a ring memory neural network is used for the attractor output signal X (j + 1) that is the output signal of the self-associative additional learning neural network 130. In the network processing 190, ring memory reading is performed.
That is, in the ring memory neutral network processing 190, the attractor output signal X (j + 1) is input to obtain the ring memory output signal Rout (1), which is stored in the learning input data storage processing 200. . Here, when the attractor output signal X (j + 1) is the prototype P (r), Rout (1) becomes the learning input data Z (r, 1).

Further, in the associative input-ring memory output coincidence detection process 210, coincidence comparison between the associative input signal X (0) and the read ring memory output signal Rout (1) is performed. Once they match, the learning input data match state is set. In the case of mismatch, a learning input data mismatch state is set.
Further, in the attractor output-learning input data coincidence state determination process 220, if the learning input data does not coincide with X (j + 1) .noteq.Rout (n), the ring memory read counter is read in the read number determination process 230. If n is less than Nmax, in the ring memory read counter n ← n + 1 process 240, n is incremented by 1, and the read Rout (1) is input and the ring memory neural network process 190 is performed again. A new ring memory output signal Rout (2) is obtained.

Further, similarly, a learning input data storage process 200, an associative input-ring memory output coincidence detection process 210, and an attractor output-learning input data coincidence state determination process 220 are performed.
When the ring memory read counter n reaches Nmax in the number-of-reads determination process 230, the spurious / attractor identification setting process 250 identifies and sets the attractor output signal as a spurious / attractor, sets the spurious / attractor state, and identifies the learning attractor. A determination process 270 is performed.

  On the other hand, if the attractor output signal X (j + 1) matches the ring memory output signal Rout (n) in the attractor output-learning input data match state determination process 220, and it is determined that the learn input data match state, the attractor In the identification processing 260, attractor identification of the attractor output signal X (j + 1) is performed as shown in Table 2.

  That is, in the attractor identification process 260, the prototype P (r) of the classification category having the number of additional learning input data nr equal to the ring memory read counter n of n ≦ Nmax and the ring memory outputs Rout (n) and X (j + 1 ), And if it matches, the attractor output signal X (j + 1) is identified as the learning attractor, and the learning attractor state is set. If the above condition is not satisfied, the attractor output signal X (j + 1) is identified as a spurious attractor, and a spurious attractor is identified. (Note that even when there are multiple prototypes having the same number of additional learning data nr, the conditions in Table 2 can be used.)

  Next, when the attractor output signal X (j + 1) is identified as the learning attractor state in the learning attractor identification determination process 270, the associative input-ring memory output coincidence detection is performed in the associative input identification process 290. If the learning input data matching state is set as a result of the process 210, the associative input signal X (0) is identified as learning input data. Here, if it matches the attractor output signal X (j + 1), it is a prototype, and if it does not match, it is identified as additional learning input data. If the learning input data mismatch state is set, the associative input signal X (0) is identified as non-learning input data.

  In learning input data reading processing 300, all additional learning input data and prototypes corresponding to the attractor output signal X (j + 1), which is a learning attractor, are read, and a series of associative loop processing is completed.

  On the other hand, if the attractor output signal X (j + 1) is identified as being in the spurious attractor state as a result of the learning attractor identification determination process 270, the associative input non-learning input data identification setting process 280 is performed. (0) is identified and set as non-learning input data (unknown associative input data), and a series of associative loop processes are all completed.

  When a new associative input signal X (0) is input, an associative loop initial setting process 110 is performed to perform the above-described series of associative loop processes.

  Next, as a second embodiment of the present invention, an associative memory device having a ring memory and using a forward additional learning neural network will be described. FIG. 2 shows an example of the configuration of the associative memory device of this embodiment.

  2 selects either the forward associative input signal X (0) from the input terminal 11 or the output signal from the backward neural network 31 by the control signal from the associative loop control processing unit 3, An input signal is input to the forward additional learning neural network 25 via the input terminal 12, and an output signal is obtained for the input changeover switch 4 to be sent to the attractor convergence detection processing unit 23 and the input signal from the input terminal 12. Are transmitted as classification category output signals, and are input to the backward neural network 31 via the input terminal 17 and the additional learning learning neural network 25, and the output signal for the input signal of the input terminal 17 is obtained, and the input changeover switch 4 and the attractor are obtained. Each of them is sent to the convergence detection processing unit 23 and sent from the output terminal 18 as an attractor output signal. Based on the control of the backward neural network 31 and the associative loop control processor 24, the input signal of the forward additional learning neural network 25 and the output signal of the backward neural network 31 are input, the attractor convergence state is detected, and the attractor convergence is detected. An attractor convergence detection processing unit 23 for sending a signal or an attractor divergence signal to the output terminal 20 and an attractor output signal obtained via the output terminal 18 or an output signal of the ring memory neural network 5 are associative loop control processing. Switching based on the control signal from the unit 24, the input change-over switch 7 to be input to the ring memory neural network 5 through the input terminal 15, and the output signal corresponding to the input signal input through the input terminal 15 are identified by the attractor Processing unit 6, learning input data memory 8, serial Control of the associative loop control processing unit 24 using the ring memory neural network 5 and the output signal from the ring memory neural network 5 to be transmitted to the virtual input identification processing unit 9 and the input changeover switch 7 for feedback as learning input data, respectively. , The learning input data memory 8 that is stored and output from the output terminal 16, and the attractor output signal from the output terminal 18 and the output signal of the ring memory neural network 5 are respectively controlled under the control of the associative loop control processing unit 24. An attractor identification processing unit 6 that inputs and identifies an attractor output signal and sends the attractor identification signal to the associative input identification processing unit 9 and the output terminal 14; a control signal from the associative loop control processing unit 24 and an attractor identification processing unit 6 Based on the attractor identification signal from the ring memory neural network. By comparing the output signal of the work 5 and the associative input signal X (0) from the input terminal 11, learning / non-learning input data of the associative input signal is identified, and a learning / non-learning input data identification signal is transmitted from the output terminal 21. The associative input identification processing unit 9, the input changeover switches 4 and 7, the attractor convergence detection processing unit 23, the associative input identification processing unit 9, the attractor identification processing unit 6, and the learning input data memory 8, respectively. 24.

  In addition, FIG. 1 which shows one structural example of the self-associative memory device in the first embodiment, FIG. 2 which shows one structural example of the mutual associative memory device in the second embodiment, and a diagram related to the prior art. 3 and FIG. 4, the functions having the same numbers are the same. Therefore, detailed description thereof will be omitted.

  Hereinafter, the operation of a series of associative loop processes including attractor convergence detection, attractor identification, associative input data identification, and learning input data reading process will be mainly described. Prior to these associative loop processes, the forward additional learning neural network 25 sets the weighting coefficient obtained by executing the additional learning process in the same manner as in the first embodiment. As additional learning processing, first, learning input data indicating a learning attractor in a prototype P (r) prepared in advance and initial learning data composed of a teacher signal corresponding to the classification category r are learned, and all correct answer outputs are obtained. Have converged initial learning.

  After that, the neighborhood test input data that has been prepared in advance is tested using the neighborhood test input data of the prototype and the teacher signal corresponding to the classification category as neighborhood test data, and the erroneous output signal is transmitted. As (r, n), after the initial learning, additional learning is performed together with the initial learning data, and all correct answer outputs are sent to these learning data and additional learning is performed until convergence. Thereafter, the test is performed again, and the additional learning is completed when all the correct outputs are obtained for the prototype neighborhood test input data. If there is an incorrect answer output, the neighborhood test data is added as additional learning input data, and the additional learning is performed again.

  Such a series of additional learning processes is performed on the prototype neighborhood test data, and is repeated until correct output is obtained for all neighborhood test input data. When the additional learning for sending the correct output is completed for the proximity test input data of all prototypes, the weighting coefficient is set and used as the forward additional learning neural network 25. Here, the learning input data composed of the prototype that was additionally learned and the vicinity test input data of the prototype that sent an incorrect output in the test is also shown in Table 1.

  Further, the reverse neural network 31 sends out all correct answer outputs for the initial learning data having an input / output relationship reverse to the initial learning data composed of the prototype and its teacher signal in the forward additional learning neural network 25. Then, learning is performed so as to converge, and the obtained weighting coefficients are respectively set.

  On the other hand, the ring memory neural network 5 uses the initial learning input data and the learning input data Z (r, n), which are the prototype P (r), used in the additional learning described above, as shown in Table 1. A teacher signal of the ring memory neural network 5 shown in the figure is prepared, learning is performed so that these learning data are output as all correct answers and converge, and the obtained weight coefficients are set in the ring memory neural network 5 and used. The learning of the ring memory neural network 5 is the same as in the case of the self-associative memory device in the first embodiment.

  Here, first, an associative loop process related to the attractor convergence detection process will be described. In the associative loop control processing unit 24, the associative loop number j = 0 is initially set, and the associative loop control signal is set to the setting of the input selector switch 4 and the internal state of the attractor convergence detection processing unit 23 and the learning input data memory 8. This is sent for initial setting of the attractor identification processing unit 6 and the associative input identification processing unit 9. The forward associative input signal X (0) is input to the input changeover switch 4 via the input terminal 11, input to the forward additional learning neural network 25 via the input terminal 12, and input to the attractor convergence detection processing unit 23. Is done. Further, the forward associative input signal X (0) is input and stored in the associative input identification processing unit 9.

  The forward incremental learning neural network 25 sends an output signal Y (0) indicating a classification category corresponding to the input X (0) as a classification category output signal, and an input terminal of the backward neural network 31 via the output terminal 13. 17 The backward neural network 31 sends an output signal X (1) to the input classification category output signal Y (0) and inputs it to the attractor convergence detection processing unit 23. The attractor convergence detection processing unit 23 sets the association loop count j = 0 from the associative loop control processing unit 24, compares the input X (0) and X (1), and if different, sets the attractor non-convergence state. Associative loop count j = 1, and an associative loop processing request is sent to the associative loop control processing unit 24. When there is an associative loop processing request, the associative loop control processing unit 24 inputs the output signal X (1) to the forward additional learning neural network 25 and the attractor convergence detection processing unit 23 via the input changeover switch 4.

  In the series of associative loop processing, when the attractor convergence detection processing unit 23 matches the input signal X (j) of the forward additional learning neural network 25 and the output signal X (j + 1) of the backward neural network 31, The attractor convergence signal is transmitted to the associative loop control processing unit 24 and the attractor convergence signal is output to the output terminal 20. At this time, if j = 0, an attractor convergence signal (One-Shot attractor convergence signal) is sent from the output terminal 20, and if j ≠ 0, an attractor convergence signal (generalized attractor convergence signal) is sent from the output terminal 20. If attractor convergence cannot be obtained even when the maximum number of associative loops J is reached, an attractor divergence signal is sent out from the output terminal 20, and the associative loop control processing unit 24 uses the attractor identification processing unit 6. The attractor divergence identification signal is transmitted from the output terminal 14 and the non-learning input identification signal is transmitted from the terminal 21 of the associative input identification processing unit 9 to complete a series of associative loop processing.

  Next, a series of associative loop processing relating to attractor identification, associative input data identification, and learning input data read processing after the attractor convergence state is detected will be described. Here, as shown in FIG. 1 in the self-associative memory device of the first embodiment and FIG. 2 of the present embodiment, the input changeover switch 7, the ring memory neural network 5, the learning input data memory 8 and the associative input. Since the configuration and connection of the identification processing unit 9 are basically the same, and the operations thereof are the same as each other, detailed description thereof will be omitted. Basically, the attractor identification processing unit 6 outputs and outputs the output signal Rout (n) sequentially read within the counter range Nmax (the maximum number of readings by category Nmax) designated from the ring memory neural network 5. There is an additional learning input data number nr equal to the ring memory read counter n when the attractor output signal X (j + 1) from the terminal 18 matches, and X (j + 1) is still the prototype P (r ) Is identified as a learning attractor, and a learning attractor identification signal is sent to the associative input identification processing unit 9. When there is no nr equal to the ring memory read counter n (where n ≦ Nmax) where X (j + 1) = Rout (n), X (j + 1) = Rout (nr) = P (r) When not established or when the ring memory read counter n> Nmax, the spurious attractor is identified and a spurious attractor identifying signal is transmitted.

When the learning attractor is identified, a series of corresponding learning input data is read from the learning input data memory 8, and the associative input identification processing unit 9 reads the forward associative input signal X (0). If it matches either the prototype or the additional learning input data, it is identified as learning input data, and in other cases, it is identified as non-learning input data, and a learning input data identification signal or non-learning input data identification signal is sent out.
As described above, the attractor convergence state is detected for the forward associative input signal X (0), and the attractor is identified based on the attractor output signal X (j + 1) corresponding to the prototype, and the forward associative input Learning / non-learning input identification of the signal X (0) and reading of a corresponding series of learning input data can be performed.

  Next, FIG. 7 shows an associative process flow when the forward associative input signal X (0) is given with respect to the mutual associative memory process shown in FIG. 2 in the second embodiment of the present invention. Here, the additional learning and learning are already completed in the forward additional learning neural network processing 430, the backward neural network processing 610, and the neural network processing for ring memory 490, respectively, as in the first embodiment. The processing flow will be described on the premise of this.

The forward associative initial setting process 400 includes the maximum associative loop count J in the associative loop count determination process 460, the category maximum read count Nmax in the ring memory read count determination process 530, and the attractor shown in Table 2 for the attractor identification process 560. Set initial parameters such as detection and identification condition settings.
The associative loop initial setting process 410 is an initial setting of the associative loop number j, j = 0, initial setting of the ring memory read counter n = 0, associative input-ring memory output coincidence detection process 510 and learning input data storage process 500. Reset internal state.
The forward associative input signal input setting process 420 sets the forward associative input signal X (0) to the forward additional learning neural network process 430, the attractor convergence detection process 440, and the associative input-ring memory output coincidence detection process 510, respectively.

The forward additional learning neural network processing 430 is a backward neural network output signal input setting process when the associative loop count j = 0 and the input forward associative input signal X (0) or the associative loop count j> 0. The output signal X (j) at the associative loop count j−1 input and set via 480 is processed, and output signals Y (0) and Y (j) are sent out, respectively.
In the backward neural network processing 610, output signals X (1) and X (j + 1) are sent to the input Y (0) and Y (j), respectively.

  When the associative loop count j = 0, the attractor convergence detection process 440 includes the input signal X (0) of the forward incremental learning neural network process 430 and the output signal X (1) of the backward neural network process 610, or the associative loop. When the number of times is j (j ≧ 1), a coincidence comparison process between the binary input signal X (j) of the forward incremental learning neural network process 430 and the output signal X (j + 1) of the backward neural network process 610 is performed. In the case of coincidence, the attractor is converged. In the case of inconsistency, the attractor is not converged. At this time, as shown in Table 3, if the number of associative loops j = 0, the one-shot attractor converges, and if the number of associative loops j ≧ 1, the generalized attractor converges.

  In the attractor convergence determination processing 450, if the attractor convergence state, the ring memory neural network processing 490 is performed. If the attractor has not converged, an associative loop number determination process 460 is performed.

In the associative loop count determination process 460, if the associative loop count j is less than the maximum associative loop count J, the associative loop count j is incremented by 1 in the j ← j + 1 setting process 470.
In the backward neural network output signal input setting process 480, X (j + 1) is set as an input of the forward additional learning neural network process 430. Thereafter, as in the case of the previous associative loop count j, forward additional learning neural network processing 430, backward neural network processing 610, attractor convergence detection processing 440 and attractor convergence determination processing 450 are performed again.

  On the other hand, if the associative loop count j = J in the associative loop count determination process 460, the associative input signal X (0) is not diverted to the attractor in the specified maximum count J associative loop process. In the associative input identification setting process 620, it is assumed that the associative input signal X (0) is non-learning input data and a divergent attractor, and all associative loop processes are completed.

  When it is determined in the attractor convergence determination processing 450 that the attractor is in a converged state, the ring memory neural network processing 490 is performed based on the attractor output signal X (j + 1) that is the output signal of the backward neural network processing 610. The process of reading the attractor identification, associative input signal identification and the corresponding learning input data by reading the ring memory via is continued. Here, these series of associative loop processes are all the same as the processes in FIG.

  One configuration example of an associative memory device using a forward additional learning neural network having a ring memory and attractor identification function in the third embodiment of the present invention will be described with reference to FIG. Note that the teacher signal used in the learning of the forward additional learning neural network 25 here is a dense binary code (for example, the integer conversion value of the binary code as the teacher signal is 0 or from some integer value). By assigning the classification category r to each of the cases (such as continuous), the spurious attractor is not generated, and the forward additional learning neural network 25 has a sparse binary output format such as “Winner takes all”. An example in which learning attractor identification and spurious attractor identification are performed in the attractor detection identification processing 26 when a teacher signal is used will be described.

  Here, for the sake of simplicity of explanation, the forward incremental learning neural network 25 has R output layer units, and its binary output signal Y (j) is expressed as One out of R or winner takes all. Assume a sparse binary encoding format called.

  5 selects either the forward associative input signal X (0) from the input terminal 11 or the output signal from the backward neural network 31 by the control signal from the associative loop control processing unit 28, An input signal is input to the forward additional learning neural network 25 via the input terminal 12, and an output signal is obtained for the input changeover switch 4 to be sent to the attractor detection / identification processing unit 26 and the input signal from the input terminal 12. As a classification category output signal, input to the backward neural network 31 via the input terminal 17, and forward additional learning neural network 25 to be transmitted to the attractor detection / identification processing unit 26, and the input signal to the input terminal 17 When an output signal is obtained and sent to the input selector switch 4 and the attractor detection identification processing unit 26, respectively. Further, based on the control of the associative loop control processing unit 28, the backward neural network 31 that is sent as an attractor output signal from the output terminal 18, the input signal and the output signal of the forward additional learning neural network 25, and the output signal of the backward neural network 31 And an attractor detection / identification processing unit 26 for detecting the attractor convergence state, identifying the converged attractor, and sending the attractor detection identification signal to the associative loop control processing unit 28 and the attractor identification signal to the output terminal 27; The attractor output signal obtained via the output terminal 18 or the output signal of the ring memory neural network 5 is switched based on the control signal from the associative loop control processing unit 28, and the ring memory neural signal is obtained via the input terminal 15. Input selector switch for input to network 5 7 and an output signal corresponding to the input signal input through the input terminal 15 are sent to the attractor match detection processing unit 29, the learning input data memory 8, the associative input identification processing unit 9, and the input changeover switch 7 for feedback, respectively. A ring memory neural network 5 and a learning input data memory 8 that stores output signals from the ring memory neural network 5 as learning input data under the control of the associative loop control processing unit 28 and outputs them from the output terminal 16. Under the control of the associative loop control processing unit 28, the attractor output signal from the output terminal 18 and the output signal of the ring memory neural network 5 are input, the matching state is detected, and the attractor matching determination signal is associative input identification processed. Attractor match detection processing unit 29 to be sent to the unit 9, and associative loop control processing unit 8 based on the control signal from the output terminal 18 and the associative input signal X (0) from the input terminal 11 to identify learning / non-learning input data of the associative input signal from the output terminal 21. Associative input identification processing unit 9 for sending a learning / non-learning input data identification signal, input selector switches 4 and 7, attractor detection identification processing unit 26, associative input identification processing unit 9 and attractor match detection processing unit 29, learning input data The associative loop control processing unit 28 controls the memory 8.

  FIG. 1 shows one configuration example of the self-associative memory device in the first embodiment, FIG. 2 shows one configuration example of the mutual associative memory device in the second embodiment, and the mutual association of this embodiment. In FIG. 5 showing one configuration example of the storage device and in FIG. 3 and FIG. 4 showing the configuration example of the prior art, the functions with the same numbers are the same. Therefore, detailed description thereof will be omitted.

As shown in Table 3, the attractor detection / identification processing unit 26, when the attractor convergence state is detected, the Hamming weight of the output signal Y (j) having the winner takes all binary representation format of the forward additional learning neural network 25. , Hamming · Weight {Y (j)}, that is, the number of 1 bits of the output signal Y (j) composed of R bits is calculated.
as a result,
Hamming ・ Weight {Y (j)} ≠ 1 （１）
Is satisfied, the output signal Y (j) of the forward additional learning neural network 25 is identified as a binary output signal of the category outside learning. That is, it is a binary output signal other than the binary teacher signal assigned to the classification category, and the attractor output signal which is the output of the backward neural network 31 is a spurious attractor, which is a spurious attractor identification.

on the other hand,
Hamming / Weight {Y (j)} = 1 (2)
If the condition is satisfied, the output signal Y (j) of the forward additional learning neural network 25 is identified as the output signal of the in-learning category, and is the same as one of the teacher signals of the forward additional learning neural network 25, and the backward neural network 31. The attractor output signal that is the output of is a learning attractor and is a learning attractor identification.

  Below, the operation | movement of these series of associative loop processes is demonstrated. Prior to the associative loop process, in the forward additional learning neural network 25, as described in the second embodiment of the associative memory device, initial learning using a prototype, output error in initial learning data and tests, and A weighting coefficient obtained by additionally learning additional learning data including the prototype test data of the prototype that has become additional learning data is set.

  Similarly to FIG. 2, the backward neural network 31 has a reverse neural network 31 with respect to initial learning data having an input / output relationship reversely arranged with respect to the initial learning data composed of the prototype and its teacher signal in the forward additional learning neural network 25. All correct outputs are sent and learned to converge, and the obtained weight coefficients are set respectively.

The forward additional learning neural network 25 sends a binary output signal having the same data structure as the backward neural network 31, and the backward neural network 31 sends an output signal having the same data structure as the forward associative input signal.
Further, in the ring memory neural network 5, as in FIG. 2, the initial learning input data and the learning input data Z (r, n), which are prototypes P (r), are used as shown in Table 1. Learning data of the memory neural network 5 is prepared, learning is performed so that the learning data becomes an all correct output and converges, and the obtained weight coefficient is set in the ring memory neural network 5.

In the associative loop process related to the attractor detection identification processing unit 26, the associative loop control processing unit 28 initially sets the associative loop count j = 0, and based on the associative loop control signal, the setting of the input selector switch 4 and the attractor detection identification. Initial settings are made for the internal state of the processing unit 26 and the learning input data memory 8, the associative input identification processing unit 9, the attractor match detection processing unit 29, and the like. When the forward associative input signal X (0) is input to the input terminal 11, the input changeover switch 4 connects the input terminal 11 and the input terminal 12, and the associative loop process related to the attractor convergence identification process is executed.
The forward associative input signal X (0) at the input terminal 11 is input to the forward additional learning neural network 25, and further input to and stored in the attractor detection identification processing unit 26 and the associative input identification processing unit 9.

  When the associative loop count j = 0, the output signal Y (0) as the classification category output signal from the forward additional learning neural network 25 and the output from the backward neural network 31 corresponding to the forward associative input signal X (0). Signals X (1) are obtained respectively. When the number of associative loops is j (where j ≧ 1), the output signal of the backward neural network 31 is sent to the input terminal 12 via the input selector switch 4 based on the control signal from the associative loop control processing unit 28. X (j) is inputted, and further, an output signal Y (j) is obtained from the forward incremental learning neural network 25 and an output signal X (j + 1) is obtained from the backward neural network 31, respectively. Each is input and stored in the identification processing unit 26.

  Further, the attractor detection and identification processing unit 26 stores the input / output signals X (j−1), Y (j−1), X stored in the forward additional learning neural network 25 and the backward neural network 31 as shown in Table 3. (j), Y (j), and X (j + 1) are compared, and the convergence state of the attractor is detected and identified. Associative loop count j (where j ≧ 1), X (j-1) = X (j), Y (j-1) = Y (j), X (j) = X (j + 1) If the relationship is satisfied, the attractor converges. When attractor convergence state is detected with j = 1, Y (j) satisfies Expression (2), and if it is a learning category, satisfies the One-Shot learning attractor identification signal, Expression (1), If the category is not learning, the One-Shot spurious attractor identification signal is output. If j ≠ 1, the generalized learning attractor identification signal is output. If the category is not learning, the generalized spurious attractor identification signal is output. Each is sent to the terminal 27. Regardless of the number of associative loops, the attractor may be identified simply as a learning attractor identification signal, a spurious attractor identification signal, or an attractor divergence identification signal.

  Further, as shown in Table 3, in the attractor detection and identification processing unit 26, if the associative loop count j is less than the maximum associative loop count J in the attractor non-converged state, the associative loop processing request signal is The attractor identification signal, which is either the learning attractor identification signal or the spurious attractor identification signal, is sent to the associative loop control processing unit 28. If the number of associative loops is the maximum number J in the attractor unconvergence state, the attractor divergence identification signal is sent to the associative loop control processing unit 28 and the output terminal 27 as the attractor divergence state.

The associative loop control processing unit 28 continues the associative loop process when the attractor unconvergence signal is input, and ends the associative loop process for attractor detection and identification when the attractor divergence identification signal is input.
When an attractor identification signal is input from the attractor detection identification processing unit 26, the associative loop process for attractor detection identification is terminated, and learning input data corresponding to learning / non-learning data identification and the attractor output signal is stored in the ring memory. The associative loop process of the ring memory reading process for reading from the neural network 5 is performed. Hereinafter, a series of these processes will be described.

Under the control of the associative loop control processing unit 28, when the one-shot and generalized spurious attractor identification signal as the attractor identification signal is input from the attractor detection identification processing unit 26, or in the case of the attractor divergence identification signal Since the attractor output signal has meaningless information, a spurious attractor identification signal or attractor divergence identification signal is transmitted from the terminal 27, and a non-learning input identification signal is transmitted from the terminal 21. Complete the associative loop process.
On the other hand, when a learning attractor identification signal such as a one-shot or generalized learning attractor is input, a classification category output signal is obtained from the output terminal 13, and based on the attractor output signal from the output terminal 18, it is used for a ring memory. A series of additional learning input data and a prototype corresponding to this from the neural network 5 are read out, and whether the forward associative input signal X (0) is unknown associative input data, prototype or additional learning input data. Perform associative input identification processing. For this reason, the attractor output signal X (j + 1) at the output terminal 18 of the backward neural network 31 is input to the ring memory neural network 5 via the input terminal 15 via the input changeover switch 7, and the learning input. The data is stored in the data memory 8 and the attractor match detection processing unit 29. For the attractor output signal X (j + 1) from the input terminal 15, the output signal Rout (1) is obtained from the ring memory neural network 5, and the attractor match detection processing unit 29, the associative input identification processing unit 9 and the learning input are obtained. Each is input to the data memory 8.

The attractor coincidence detection processing unit 29 compares the attractor output signal X (j + 1) with the output Rout (1). If they do not coincide, the ring memory read counter n = n + 1 is incremented by 1 (however, the initial value (Value n = 0), and the input changeover switch 7 is switched under the control of the associative loop control processing unit 28, and Rout (1) is fed back to the ring memory neural network 5 via the input terminal 15.
Until the attractor output signal X (j + 1) and the output Rout (n) match within the range in which the ring memory read counter n in the attractor coincidence detection processing unit 29 reaches the pre-categorized read maximum count number Nmax. The above-described series of feedback read processing is repeated. The category-specific maximum read count number Nnax is set to the maximum value Nmax in the category-specific additional learning input data number nr shown in Table 1. When the attractor output signal X (j + 1) matches the output Rout (n), all learning input data corresponding to the classification category is read out and stored in the learning input data memory 8.

  In addition, a ring memory read completion signal is output to the associative input identification processing unit 9. In the associative input identification processing unit 9, if the associative input signal X (0) coincides with any of the additional learning input data read during this period, the associative input signal X (0) is converted into the additional learning input data or the prototype. , The learning input data is identified, and if it does not match, the non-learning input data is identified and transmitted from the output terminal 21. Further, an all process completion signal is sent to the associative loop control processing unit 28. When the associative loop control processing unit 28 receives this signal, all the series of processing is completed, and a new associative input signal is received. When a new forward associative input signal X (0) is input to the input terminal 11, the number of associative loops j = 0 is set again, and the above-described series of associative loop processing is performed.

  In the third embodiment of the present invention, a case has been described in which learning internal / external category identification processing is performed using the winner take all or one out of R output format as the binary output format of the forward incremental learning neural network 25. However, the present invention is not limited to this binary output format, as long as the learning internal / external category can be identified from the output signal of the forward incremental learning neural network 25 based on the binary output format. Also, for example, the details are described in Japanese Patent Application No. 2004-236466 “Interassociative Memory Device and Software”, but all dense binary codes, which are binary teacher signals, are converted into consecutive serial numbers. , A classification category number region composed of serial numbers from 0 to the maximum classification category number (= R-1) is provided, and the binary output signal Y (j) of the forward incremental learning neural network 25 is converted into a classification category number for comparison processing. Using a teacher signal in which a classification category number area composed of classification category numbers consecutive from classification category numbers equal to or greater than a specific integer value is provided, and these are associated with binary codes. A learning internal / external category identification method that performs learning and converts the binary output signal Y (j) into classification category numbers and performs comparison processing may be used. When a binary teacher signal in such an output representation format is used, since a spurious attractor does not occur, the attractor detection identification processing unit 26 only needs to detect the attractor convergence state.

  That is, a binary output signal having a distributed output format of a binary code that is equal to or greater than the minimum number of bits required to represent the above category category number area may be used. In particular, when the number of binary teacher signals or the number of classification categories R is a power of 2 (R = 2 ** Q, a complete packing code), a non-learning category is generated by using the required minimum number of bits. It never converges, it always converges to a learning attractor or generalized learning attractor, does not generate One-Shot and generalized spurious attractors, and realizes a very good characteristic, and makes attractor identification easy.

The attracted attractors were identified as One-Shot learning attractor, generalized learning attractor, One-Shot spurious attractor, and generalized spurious attractor. You may identify.
As described above, the attractor convergence state is detected for the forward associative input signal X (0), the attractor is identified, the classification category output signal Y (j) is obtained, and the attractor output signal X corresponding to the prototype is obtained. Based on (j + 1), learning / non-learning input identification of the forward associative input signal and reading of a series of corresponding learning input data can be performed.

  Next, FIG. 8 shows an associative process flow when the associative input signal X (0) is given with respect to the mutual associative memory process shown in FIG. 5 in the third embodiment of the present invention. Here, in the forward additional learning neural network processing 730, the backward neural network processing 610, and the neural network processing for ring memory 490, processing is performed on the assumption that additional learning and learning have already been completed as described above. The flow will be described.

The forward associative initial setting process 700 performs initial parameter settings such as the maximum associative loop number J in the associative loop number determination process 460 and the attractor detection and identification condition settings shown in Table 3 in the attractor identification process 740.
In the associative loop initial setting process 710, the initial setting of the associative loop count j is j = 0, the initial setting of the ring memory read counter n = 0, the associative input-ring memory output match detection process 510, and the learning input data storage process 500. Reset internal state.
In the forward associative input signal input setting process 720, the forward associative input signal X (0) is input to the forward additional learning neural network process 730, the attractor detection identification process 740, and the associative input-ring memory output coincidence detection process 510, respectively.

The forward additional learning neural network processing 730 is a backward neural network output signal input setting process when the associative loop count j = 0 and the input forward associative input signal X (0) or the associative loop count j> 0. The output signal X (j) input and set via 480 is processed, and output signals Y (0) and Y (j) are sent out, respectively.
In the backward neural network processing 610, output signals X (1) and X (j + 1) are sent to the input Y (0) and Y (j), respectively.

  The attractor detection identification process 740, when the association loop count j = 0, is the input signal X (0), the output signal Y (0) of the forward additional learning neural network process 730, and the output signal X of the backward neural network process 610. (1) or in the case of the association loop count j, the input signal X (j), the output signal Y (j) of the forward additional learning neural network processing 730, and the output signal X (j + 1) of the backward neural network processing 610 ) Is used to detect attractor convergence. If X (j) and X (j + 1) match, the attractor convergence state, and if the association loop count j = 0 at this time, the One-Shot attractor convergence state, the association loop count j ≧ 1 In this case, the generalized attractor converges. If they do not match, the attractor is not converged.

  Furthermore, in the attractor convergence determination processing 450, if the attractor convergence state, the ring memory neural network processing 490 is performed. If the attractor has not converged, an associative loop number determination process 460 is performed.

In the associative loop count determination process 460, if the associative loop count j is less than the maximum associative loop count J, the associative loop count j is incremented by 1 in the j ← j + 1 setting process 470.
In the backward neural network output signal input setting process 480, X (j + 1) is set as an input of the forward additional learning neural network process 730. Thereafter, as in the case of the previous associative loop count j, the forward additional learning neural network processing 730, the backward neural network processing 610, the attractor detection identification processing 740, and the attractor convergence determination processing 450 are performed again.

  On the other hand, if the associative loop count j = J in the associative loop count determination process 460, the associative input signal X (0) is not diverted to the attractor in the specified maximum count J associative loop process. In the associative input identification setting processing 620, the associative input identification signal X (0) sends a non-learning input identification signal and an attractor divergence identification signal to complete all associative loop processing. .

In the attractor determination and identification process 740, if it is determined that the attractor is in a converged state, it is further identified whether it is a learning attractor or a spurious attractor.
In the attractor determination and identification process 740, the hamming weight of the binary output code that is the output signal Y (j) of the forward additional learning neural network process 730 is calculated. Since it is an output signal, it is identified as a spurious attractor. When Expression (2) is satisfied, it is identified as a learning atraqua because it is an output signal in the learning category.

If it is determined in the learning attractor determination processing 750 that the attractor output signal X (j + 1), which is the output signal of the backward neural network processing 610, is spurious attractor identification, the associative input identification setting processing 630 is performed. Then, a spurious attractor identification signal and a non-learning input identification signal are transmitted to complete the entire series of associative loop processing.
On the other hand, if it is determined that the learning attractor is identified, the associative input signal identifying process by ring memory reading via the ring memory neural network process 490 and the corresponding learning input data reading process are continued as follows.

In the ring memory neural network processing 490, an attractor output signal X (j + 1) as a learning attractor is input to obtain a ring memory output signal Rout (1), which is stored in the learning input data storage processing 500. To do.
Further, in the associative input-ring memory output coincidence detection process 510 for associative input identification, the associative input signal X (0) and the ring memory output signal Rout (1) are compared for coincidence. Once they match, it is clear that the input data is learning input data, and additional learning input data or a prototype, so a learning input data matching state is set.

  In the attractor output-learning input data match state determination process 520, a match determination is made between the attractor output signal and the ring memory output Rout (1). If there is a mismatch, the ring memory read counter n ← n + 1 process 540 is entered. Then, n is incremented by 1, and the ring memory neural network processing 490 is performed again to obtain a new ring memory output signal Rout (n). Further, a learning input data storage process 500, an associative input-ring memory output coincidence detection process 510, and an attractor output-learning input data coincidence state determination process 520 are performed.

When the attractor output signal X (j + 1) matches the ring memory output signal Rout (n) in the attractor output-learning input data matching state determination process 520, the attractor output signal X (j + 1), which is a learning attractor, is obtained. Since all the corresponding additional learning input data and prototype P (r) have been read out, associative input-ring memory output coincidence detection processing 510 is set in the associative input identification process 590, so that the learning input data coincidence state is set. For example, the associative input signal X (0) is identified as learning input data. Further, if it matches the attractor output signal, it is a prototype, and if it does not match, it is identified as additional learning input data. If the learning input data mismatch state is set, the associative input signal X (0) is identified as non-learning input data.
In learning input data reading processing 600, all additional learning input data and prototypes corresponding to the attractor output signal X (j + 1), which is a learning attractor, are read, and a series of associative loop processing is completed.

  When a new associative input signal X (0) is input, an associative loop initial setting process 710 is performed, and the above series of associative loop processes are performed.

  In the associative memory device of the above-described embodiment, when the number of learning attractors that are prototypes of the self-associative neural network and the forward additional learning neural network corresponding to the storage amount is very large, that is, the required storage amount of the associative memory is very large. Even if it is large, by increasing the number of intermediate layer units of the multilayer neural network, it is possible to reliably converge additional learning correctly and to realize associative memory. Therefore, the associative characteristics of the storage capacity can be dramatically improved as compared with the conventional associative memory device.

  In the conventional self-associative memory, the learning attractor that can be stored due to the limitation of the storage capacity is limited to a representative one, and there is a restriction that the learning attractors are not much correlated with each other. In addition to this typical learning attractor, the self and mutual associative memory devices can store a large amount of complex input data having a large correlation similar to them as additional learning input data. The reverse associative input signal can be converged to the learning attractor.

In the embodiment of the present invention, the case where a three-layer neural network is used has been described, but a multilayer neural network having three or more layers may be used. Furthermore, the present invention is not limited to a multilayer neural network, and may be a neural network trained using a binary teacher signal having the same input / output format.
Also, instead of using a sparse binary output code such as Winner Takes all for a classification category, a dense distributed binary code, for example, an output binary in which a classification category is assigned to all binary codes that can be represented. High generalization ability using code (complete packing code) and output binary code (consecutive allocation code) that assigns a classification category to a binary code converted into a continuous integer value within a certain range A forward incremental learning neural network may be applied. As a result, processing can be performed without generating spurious attractors.

As described above, the conventional self-associative memory has a very small data storage capacity. Further, there is a problem that the associative ability cannot be increased depending on the number of learning attractors and the distance distribution of the learning attractors.
In addition, the conventional associative memory device can increase the data storage capacity, but the associative characteristics are not so excellent, such as a large number of spurious attractors. However, the attractor output signal from the backward multi-layer neural network cannot identify the desired learning attractor or spurious attractor or associative input. Therefore, malfunctions cannot be avoided in the application processing system using the attractor output signal.

  On the other hand, in the associative memory device of the present invention, the number of intermediate layer units was increased according to the large number of learning attractors to be stored, and the additional learning was performed by converging to obtain all correct answer outputs. By introducing an additional learning neural network with a high generalization ability, the spurious attractor generation is greatly reduced and the attracting force of the attractor is improved. Can learn learning attractors. Also, by introducing a ring memory storing prototypes and additional learning input data, learning / non-learning input data identification of associative input signals, attractor identification between learning attractors and spurious attractors in the readout loop processing, and It is possible to dynamically read out all learning input data corresponding to the attractor output signal.

As described above, the structure and characteristics of the self and the associative memory are very excellent, a free design according to various applications is possible, and the reliability is also excellent. Therefore, the conventional self-associative memory device has various practical restrictions on not only the storage capacity but also the attracting force of the attractor and the generation of many spurious attractors, but these problems can be solved. .
In addition to the excellent associative characteristics and large-capacity storage capability, it also has a category classification capability, attractor identification capability, associative input identification capability, and a learning input data readout function corresponding to converged attractor output signals. It can be applied to a very wide range of application processing systems.

  Due to the above-mentioned wide effects, the self-associative memory device having the ring memory of the present invention and using the additional learning neural network can be applied to security systems such as security management systems and unauthorized access detection, and biometrics. Network failure detection system such as various high-performance pattern recognition and application to recognition of large-capacity image data degraded by a lot of noise, application to high-speed data retrieval, etc. It has characteristics that can be applied to a wide range of applications.

1 is a configuration example of a self-associative memory device using a self-associative additional learning neural network having a ring memory storing attracting learning input data and an attractor identification function in the first embodiment of the present invention. . It is one structural example of the mutual associative memory device using the forward additional learning neural network which has the ring memory and attractor identification function which memorize | stored the additional learning input data in the 2nd Embodiment of this invention. It is one structural example of the self-associative memory device by the self-associative neural network in the conventional system. It is one structural example of the mutual associative memory device using the advance and update neural network in a conventional system. It is one structural example of the mutual associative memory device using the forward additional learning neural network which has the ring memory and the attractor identification function which memorize | stored the additional learning input data in the 3rd Embodiment of this invention. It is an associative loop process flow figure of the self-associative memory with respect to the associative input signal in the 1st Embodiment of this invention. It is an associative loop process flow figure of the mutual associative memory with respect to the forward associative input signal in the 2nd Embodiment of this invention. It is an associative loop processing flow figure of the mutual associative memory with respect to the forward associative input signal in the 3rd Embodiment of this invention.

Explanation of symbols

1 Additional learning neural network for self-association 2 Attractor convergence detection processing unit 3 Associative loop control processing unit 4 Input changeover switch 5 Neural network for ring memory 6 Attractor identification processing unit 7 Input changeover switch 8 Learning input data storage memory 9 Associative input identification processing unit 10 Input changeover switch 11 Associative input signal terminal 12 Neural network input terminal 13 Neural network output terminal 14 Attractor identification output terminal 15 Ring memory input terminal 16 Learning data output terminal 17 Reverse neural network input terminal 18 Reverse neural network output terminal 19 Reverse associative input terminal 20 Attractor convergence detection output terminal 21 Associative input identification output terminal
22 Attractor detection and identification processing unit
23 attractor convergence detection processing unit 24 associative loop control processing unit 25 forward additional learning neural network 26 attractor detection identification processing unit 27 attractor detection identification output terminal 28 associative loop control processing unit 29 attractor coincidence detection processing unit 31 backward neural network 32 forward neural network 33 Self-associative neural network 34 Attractor convergence detection processing unit 35 Associative loop control processing unit 100 Associative initial setting processing 110 Associative loop initial setting processing 120 Associative input signal input setting processing 130 Self-associative additional learning neural network processing 140 Attractor convergence detection processing 150 attractor convergence determination processing 160 associative loop number determination processing 170 associative loop number j ← j + 1 setting processing 180 additional learning neural network output for self-association Number input setting processing 190 Ring memory neural network processing 200 Learning input data storage processing 210 Associative input-ring memory output coincidence detection processing 220 Attractor output-learning input data coincidence state determination processing 230 Ring memory read count determination processing 240 Ring memory read counter n n ← n + 1 setting processing 250 spurious attractor identification setting processing 260 attractor identification processing 270 learning attractor identification determination processing 280 associative input non-learning input data identification setting processing 290 associative input identification processing 300 learning input data reading processing 310 associative input Identification setting processing 400 Forward association initial setting processing 410 Associative loop initial setting processing 420 Forward association input signal input setting processing 430 Forward additional learning neural network processing 440 Attractor convergence detection processing 4 0 attractor convergence determination processing 460 associative loop count determination processing 470 associative loop count j ← j + 1 setting processing 480 reverse neural network output signal input setting processing 490 ring memory neural network processing 500 learning input data storage processing 510 associative input-ring memory Output match detection processing 520 Attractor output-learning input data match status determination processing 530 Ring memory read count determination processing 540 Ring memory read counter n n ← n + 1 setting processing 550 Spurious attractor identification setting processing 560 Attractor identification processing 570 Learning attractor Identification determination processing 580 Association input non-learning input data identification setting processing 590 Association input identification processing 600 Learning input data read processing 610 Reverse neural network processing 620 Association input identification setting processing 63 0 associative input identification setting process 700 forward associative initial setting process 710 associative loop initial setting process 720 forward associative input signal input setting process 730 forward additional learning neural network process 740 attractor detection identification process 750 learning attractor identification determination process

Claims (12)

  A learned neural network means 1 for self-association, a learned neural network means 5 for ring memory that has learned learning data arranged in a ring gradient consisting of learning input data of the learned neural network means 1 for self-association, and the self-association Attractor convergence detecting means 2 for detecting an attractor convergence state with respect to an associative input signal input to the learned neural network means 1 and the learning input data stored in the ring memory learned neural network means 5 are sequentially read out. Attractor identifying means 6 for identifying an attractor as a spurious attractor or a learned attractor by at least a coincidence comparison with the attractor output signal converged on the attractor output from the self-associative learned neural network means 1 Autoassociative storage device characterized by having even without.   A forward-learned neural network means 25 for mutual association and an output signal corresponding to an input signal from the forward-learned neural network means 25 are fed back to the forward-learned neural network means 25 and an attractor output signal in an attractor converged state is transmitted. The backward learned neural network means 31 for learning the learning data obtained by reversing the learning input data and the teacher signal as learning data of the forward learned neural network means 25, and the forward learned neural network means 25 The learned neural network means 5 for the ring memory that has learned the learning data in which the learning input data is arranged in a ring gradient, and the atlas for the associative input signal input to the forward learned neural network means 25 Attractor convergence detecting means 2 for detecting the data convergence state, and the learning input data stored in the learned neural network means 5 for the ring memory are sequentially read out, and the attractor output from at least the backward learned neural network means 31 A mutual associative memory device comprising at least an attractor identifying means 6 for identifying an attractor as a learning attractor or a spurious attractor of the attractor output signal by comparing with an attractor output signal converged to   A forward-learned neural network means 25 for mutual association and an output signal corresponding to an input signal from the forward-learned neural network means 25 are fed back to the forward-learned neural network means 25 and an attractor output signal in an attractor converged state is transmitted. The backward learned neural network means 31 for learning the learning data obtained by reversing the learning input data and the teacher signal, which are learning data of the forward learned neural network means 25, and the forward learned neural network means 25 learned neural network means 5 for ring memory that has learned learning data in which the 25 learning input data are arranged in a ring gradient, and an atlas for the associative input signal input to the forward learned neural network means 25 Whether the attractor learning signal or the spurious attractor of the attractor output signal converged to the attractor output from the backward learned neural network means 31 is detected using the output signal of the forward learned neural network means 25. The learning input sequentially read from the learned neural network means 5 for the ring memory based on the attractor detection identifying means 26 for identifying the attractor and the attractor output signal output from the backward learned neural network means 31 At least an attractor coincidence detecting unit 29 that performs at least a coincidence comparison between the data and the attractor output signal and controls reading of the learned input data from the learned neural network unit 5 for the ring memory. Virtual storage device.   4. The associative memory device according to claim 1, wherein said learning input data from said learned neural network means for ring memory 5 and said associative input signal are at least compared, and said associative input signal An associative storage device comprising associative input identifying means 9 for identifying learning / non-learning input data.   5. The associative memory device according to claim 1, further comprising learning input data storage means 8 for sequentially reading out and storing said learning input data from said learned neural network means 5 for ring memory. An associative memory device.   A learned neural network process for self-association 130, a learned neural network process for ring memory 190 that learns learning data arranged in a ring gradient consisting of learning input data of the learned neural network process for self-association 130, and the self-association An attractor convergence detection process 140 for detecting an attractor convergence state with respect to an associative input signal inputted to the learned neural network process 130, and the learning input data stored in the learning neural network process 190 for the ring memory, The attractor is identified as a learning attractor or a spurious attractor by at least comparing with the attractor output signal converged on the attractor output from the self-associative learned neural network processing 130. Auto-associative memory software is characterized by having at least a tractor identification process 260.   A forward-learned neural network process 430 for mutual association and an output signal corresponding to the input signal from the forward-learned neural network process 430 are fed back to the forward-learned neural network 430 and an attractor output signal in an attractor converged state is sent out. The backward learned neural network processing 610 for learning the learning data obtained by reversing the learning input data and the teacher signal as learning data of the forward learned neural network processing 430, and the forward learned neural network processing 430 The neural network processing 490 for ring memory that has learned the learning data in which the learning input data is arranged in a ring inclination, and the associative input signal input to the neural network processing 430 that has been forward-learned Attractor convergence detection processing 440 for detecting a tractor convergence state and the learning input data stored in the ring memory learned neural network processing 490 are sequentially read out, and at least the attractor output from the backward learned neural network processing 610 And an attractor identification process 560 for at least identifying the attractor as a learning attractor or a spurious attractor of the attractor output signal by making a coincidence comparison with the attracted output signal.   A forward-learned neural network process 730 for mutual association and an output signal corresponding to the input signal from the forward-learning neural network process 730 are fed back to the forward-learned neural network process 730 and an attractor output signal in the attractor convergence state is transmitted. The backward-learned neural network process 610 for learning the learning data obtained by reversing the learning input data and the teacher signal as learning data of the forward-learned neural network process 730, and the forward-learned neural network process 730 The learned neural network processing 490 for learning the ring data in which the learning input data is arranged in a ring gradient, and the associative input input to the forward learned neural network processing 730 An attractor convergence state with respect to the signal, and using the output signal of the forward learned neural network processing 730, the learning attractor or spurious of the attractor output signal converged on the attractor output from the backward learned neural network processing 610 is detected. The learning is sequentially read from the learned neural network processing 490 for the ring memory based on the attractor identification processing 740 for identifying the attractor of the attractor and the attractor output signal sent from the backward learned neural network processing 610. Attractor output-learning input data coincidence state that performs at least coincidence detection between the input data and the attractor output signal and controls reading of the learned input data from the learned neural network processing 490 for the ring memory Mutual Associative Memory software characterized by having at least a determination process 520.   9. The associative memory software according to claim 6, 7, and 8, wherein at least a comparison between the learning input data from the learned neural network processing 730 for the ring memory and the associative input signal is performed, and the associative input signal is detected. Associative memory software characterized by having associative input identification processing 590 and 290 for identifying learning / non-learning input data.   A learning input data storage process 200 for sequentially reading out and storing the learning input data from the learned neural network processes 190 and 490 for the ring memory in the associative memory software according to claim 6, 7, 8, and 9. 500 associative memory software.   In the associative learned neural network means 1 and 25 according to claims 1 to 5, the neural network is made to learn initial learning data prepared in advance comprising a prototype corresponding to a classification category and its teacher signal. After initial learning, testing is performed using prototype neighborhood test data prepared in advance consisting of prototype neighborhood test input data and its teacher signal, and prototype neighborhood test input data resulting in incorrect answer output is used as additional learning input data. Performing additional learning with the prototype, repeating the prototype neighborhood test data additional learning process until all of the prepared prototype neighborhood test input data are correct output, and setting the weighting coefficient obtained associative learned neural network Using means 1 and 25 Features and the content addressable memory. As the associative learned neural network processes 130 and 430 according to claims 6 to 10, initial learning is performed by causing a neural network to learn initial learning data prepared in advance consisting of a prototype corresponding to a classification category and a teacher signal thereof. After that, the prototype neighborhood test input data consisting of the prototype neighborhood test input data and the teacher signal is used to perform the test, and the prototype neighborhood test input data resulting in incorrect answer output is used as additional learning input data together with the prototype. Additional learning is performed, and the prototype neighborhood test data additional learning processing is repeated until all of the prepared prototype neighborhood test input data are output as correct answers, and the associated learned neural network processing 130 in which the obtained weighting factor is set. 430, Associative Memory software characterized by using a 30.

Images