WO1992012497A1 - Neural network with binary operators and methods of implementation - Google Patents

Abstract

A neural network consisting of neurones comprising input synapses connected to a summation operator, said neurones being interconnected, is characterized in that it comprises neurones having entirely binary operation and that the summation operator provides a 'NOR' logic function so that its output is active only if all its inputs are inactive. The neurones are grouped in mutual exclusion domains which comprise several neurones of a first type (N1, N2,...NN) and a single neurone of a second type (G1). In each mutual exclusion domain, the synapses of the neurone of the second type (G1) receive the output signals from the neurones of the first type in the domain concerned and also the output signals from the neurones of the second type in the domains interconnected to the domain concerned, and in each mutual exclusion domain the synapses of the neurones of the first type receive the output signals from the neurones of the first type in the domain concerned, the output signals from the neurones of the first type in the domains interconnected with the domain concerned, at least an inhibition signal from the neurones of the second type in the domains interconnected with the domain concerned, and external signals.

Description

 Neural network with binary operators and methods for its realization.

The present invention relates to associative memories constituted by neural networks. Such networks are generally made up of artificial neurons comprising input synapses connected to a summation operator followed by a non-linear element, said artificial neurons being interconnected with one another.

There are two types of existing neural networks implanted on silicon: layered networks, and completely connected networks, called associative memories because of their ability to reconstruct a complete configuration of input signals from a fraction of this. configuration. All these networks presently present two major drawbacks: on the one hand, their operation involves the implementation of relatively complicated algorithms; on the other hand, their effectiveness depends on the precision of the "force" of the synapses connecting the neurons to each other, the volume of material required being all the more important when one wants a finer resolution. These networks, in which the synapses are produced by a combination of multiplications and summations, analogical or digital, generally do not integrate the learning mechanism: either because the materialized model does not allow it (absence of "local law "synapses evolution), either by lack of space on the silicon, because of the important surface occupied by the synapses.

The present invention proposes to provide a neural network which does not have the aforementioned drawbacks.

To this end, the subject of the invention is a neural network comprising neurons comprising input synapses connected to a logical operator, said neurons being interconnected with each other, characterized in that: - it is made up of fully binary neurons;

- the summation operator performs a "NON-OR" logic function such that its output is only active if all of its inputs are at the inactive level;

- the neurons are grouped into areas of mutual exclusion which include several neurons of a first type N and a single neuron of a second type G;

- in each area of mutual exclusion, the synapses of the neuron of the second type G respectively receive the output signals of the neurons of the first type N of the domain considered and the output signals of the neurons of the second type G of the domains interconnected with the domain considered; and - in each mutual exclusion domain, the synapses of the neurons of the first type N respectively receive the output signals of the neurons of the first type N of the domain considered, the output signals of the neurons of the first type N of the domains interconnected with domain considered, at least one inhibition signal coming from neurons of the second type G of the domains interconnected with the domain considered and external signals.

It is therefore a network of binary neurons linked together by binary synapses functioning in all or nothing, that is to say that they are active or blocked.

According to one embodiment of the invention, the N type neurons are connected by a synapse to the output of each of the G neurons from the domains interconnected to their domain.

According to a second embodiment, each neuron of the second type G comprises a circuit generating an inhibition signal H, said circuit being controlled by the output signal of the type G neuron considered and by the output signals of type G neurons from the domains interconnected to the domain considered. The inhibition signal is transmitted to each neuron of type N by a synapse connected to the output of the circuit generating the inhibition signal H of the neuron of type G of the domain considered. This second embodiment makes it possible to limit the number of synapses necessary to effect the inhibition of an N neuron by a G neuron from an external domain. According to another characteristic of the invention, each neuron of the second type G comprises a circuit generating a learning command (A1, A2) controlled by the output signal of this neuron, the neurons of the first type N comprise interruptible synapses provided with a control circuit and a circuit generating a blocking signal Z and a learning signal C, receiving the learning command (A1, A2) supplied by the neuron of the second type of domain considered, and an INIT initialization signal. In each neuron of the first type N, the abovementioned blocking Z and learning signals C are sent to all the control circuits of the interruptible synapses.

Thanks to this arrangement, learning, that is to say the control of the modification of the synapses, is carried out in a simple manner.

Advantageously, the learning command is a two-bit reflected binary code.

The invention relates to a method for producing an associative memory constituted by a neural network according to the invention, this method being characterized in that a neural network is connected, connected by permanent synapses according to a determined configuration. A non-modifiable memory is thus produced.

The invention also relates to another method for producing an associative memory constituted by a neural network according to the invention, this method being characterized in that a network is made up of interruptible synapses controllable by an external system. The interruptible synapses can be constituted by fuses, switching elements controlled by a rocker or switching elements controlled by a memory cell.

The invention also relates to a method for producing an associative memory constituted by a neural network according to the invention, characterized in that, by the action of the initialization signal, all the interruptible synap¬ its are set to active state and that unsupervised learning is then carried out. The invention will be better understood with the aid of the description which follows, given solely by way of examples and made with reference to the appended drawings in which:

- Figure 1 A is the basic diagram of a neuron according to the invention and Figure 1 B represents the response curve of the amplifier-inverter stage;

- Figure 2 shows an electrical embodiment of the neuron of Figure 1;

- Figure 3 is a symbolic representation of this neuron;

- Figure 4 A is an exemplary embodiment of a modifiable synapse with its control circuit and Figure 4 B its symbolic representation;

FIG. 4 C represents an example of the creation of a permanent synapse and FIG. 4 D is its symbolic representation;

- Figure 5 shows the Petri graph of the triggering of the modifiable synapse of Figure 4 A; - Figure 6 A is the electrical diagram of the learning control circuit and Figure 6 B its symbolic representation;

- Figure 7 illustrates the learning sequences;

- Figure 8 is the complete symbolic diagram of a neuron of the first type N;

- Figure 9 is the complete block diagram of a type G neuron; FIG. 10 A is the electrical diagram of the circuit for generating the inhibition signal and FIG. 10 B is the diagram of the circuit for generating the learning signals;

- Figure 11 is a block diagram of a mutual exclusion domain;

- Figure 12 is an explanatory diagram representing a field of mutual exclusion with three elements;

- Figures 13 A and 13 B show two embodiments of a mutual exclusion network; - Figure 14 is the symbolic representation of a network consisting of two areas of mutual exclusion;

FIG. 15 is the timing diagram representing the changes in the signals between two areas of mutual exclusion before learning;

- Figure 16 shows the matrix of connections of an exemplary embodiment of a neural network constituting an associative memory before its learning phase; - Figure 17 shows the matrix of connections of the same network after learning;

- Figure 18 describes the propagation of constraints in a mutual exclusion network;

- Figure 19 shows an example of a tree network; - Figure 20 is a timing diagram showing the evolution of the learning signals during a first type learning;

- Figure 21 is a timing diagram showing the evolution of learning signals during a second type of learning;

- Figure 22 describes the operation of a modifiable synapse; and

- Figures 23 A and 23 B illustrate the operation of the learning control circuit.

In the description which follows, it will be considered that a logic signal can take two values: OV for the "inactive" state ("0"), and + V (value of the supply voltage) for the " active "(" 1 "). The sign "/" after a signal name corresponding to a logical inversion. For example, the signal "C" being active at "1", its inverse "C /" is active at "0".

Figure 1A is the block diagram of a neuron according to the present invention. It comprises several input synapses E1 to EN whose signals are sent on a summing circuit Σ; the output of this summing circuit is sent to a gain inverting amplifier - A, A being greater than 1, which has a VTH threshold as can be seen on the curve of FIG. 1 B which shows the output S of this inverting amplifier depending on the input signal E. The logic signals used in this circuit can take the value + V for the active state and the value zero for the inactive state. The characteristics of the inverting amplifier fulfill the following condition: V --- A <VTH <+ V

The excursion of the output signal is limited to a range between OV and + V. The value of the gain A and that of the threshold VTH are such that an active input (= + V) is enough to put the output in the inactive state (= 0V). When all the inputs are inactive, the output is active. The neuron therefore behaves like a logical "NO-OR" element.

FIG. 2 represents an example of an electrical device advantageously performing this function with a MOSFET transistor, P channel. Each permanent synapse consists of a simple diode. Figure 3 is a proposed symbolic representation of the neuron.

The different synapses of the neurons according to the invention can be either permanent, or modifiable or interruptible, which means that the synapse can take two states, valid or invalid. A signal relating to a synapse is only taken into account by the summing-inverting stage if this synapse is valid. In the initial state the synapses are all valid.

Consequently, a neuron of this type is in the active state if all the afferent signals having their valid synapses are inactive.

FIGS. 4 A and 4 B represent an exemplary embodiment of a modifiable synapse and of its control circuit. The circuit shown in FIG. 4 A consists of five NAND logic circuits 11 to 15, an inverter circuit 16 and a controlled switch 17 whose output is connected to a diode 18. The control circuit receives three signals C /, E and Z / and outputs an X signal, as symbolized in FIG. 4 B. The signal E is one of the afferent signals of the neuron, the signal C / is a signal for controlling learning and the signal Z / is an initialization signal. The signal X supplied by this synapse is sent to the inverting summing stage which has been described above.

FIGS. 4 C and 4 D represent an example of embodiment of a non-modifiable or permanent synapse; it can be constituted by a diode 21 and represented symbolically as shown in FIG. 4 D. The triggering of the synapse of figure 4 A corresponds to the sequence described by the Petri graph of figure 5. At the start, the signal Z places the synapse in the valid state, that is to say that the signal output of the synapse is the copying of the related signal E connected to its input. The transition of this synapse from the valid state to the invalid state takes place according to the following sequence:

- initial state: valid synapse (state 1); - C active: the synapse temporarily goes to the invalid state (state 2);

- if the signal C returns to the inactive state while E is inactive or Z is active, the synapse rede¬ comes valid (state 1); - if signal C returns to the inactive state while E is active and Z is inactive, the invalid state becomes definitive (state 3). Learning is then carried out.

FIG. 6 A describes a circuit for controlling the learning of a neuron of type N. This circuit receives the input signal INIT, the logic signals Al and A2, and the output N (the state) of the neuron possibly by via a low-pass filter 31 in order to guarantee against transient states at the time of competition between the neurons. It delivers as output the logic signal C for controlling learning, and the logic signal Z, signals which are distributed over all the modifiable synapses of the neuron.

The logical equation of signal C is as follows: C = (A1.A2 / .V) + T with: V ≈ T /. (V + INIT) (V = 1: virgin neuron)

T = (Al.N) + (A2.T)

The logical equation of the signal Z / is as follows:

Z / = (INIT + V) / The learning command sent to the neuron synapses is active when (A1, A2) = (1.0), if the neuron is blank. It is then a type of learning type 1. The learning command is also activated when Al = 1, if the neuron is active (type 2 learning). In this case, the command remains maintained after the deactivation of Al, until A2 returns to zero. FIG. 7 indicates the actions carried out by the learning control circuit as a function of the signals A1 and A2.

FIG. 8 represents the complete diagram of a neuron of type N, integrating the elements described above: permanent synapses, modifiable synapses, amplifier-inverter stage similar to the basic diagram, circuit for controlling learning.

Each exclusion domain comprises a large number of neurons of a first type N which has just been described above; each exclusion domain also includes a second type G neuron, FIG. 9 of which represents the complete block diagram.

According to the invention, this type G neuron receives a synapse from each of the type N neurons of its exclusion domain, namely NI, N2 ..., NN and synapses originating from the same type G neurons of the others. exclusion fields, namely G2, G3 ..., GN. Synapses originating from type N neurons are sent to a first summing circuit 21 followed by an inverting amplifier 22 of the type described above; similarly, the synapses coming from type G neurons of the external domains are sent to a second summator 23, the output of which is sent to an inverting amplifier 24. The signals supplied by these two summing amplifiers are sent to an AND logic circuit which provides at the output a signal G. This signal G is sent to a circuit 26 generator of learning signals A1 and A2 and on a circuit 27 generator of an inhibition signal H which also receives the output signal from the adder 24. FIG. 10 A represents an exemplary embodiment tion of the circuit generating the inhibition signal H. The output signal G of the logic circuit 25 is sent to an inverter 31 whose output is sent to a timer circuit 32 introducing a time delay T2; the circuit coming from the summing amplifier 24 of the signals of the synapses connected to the neurons G of the other domains is sent to an inverter 33 whose output signal is sent directly and via a timer circuit 34 introducing a time delay Tl on an AND logic circuit 35 whose output is sent to a logic circuit 36 which also receives the signal from the timing circuit 32 and provides at its output the inhibition signal H.

It follows that the signal H is obtained by performing a logical "AND" taking into account three signals: the logical sum of the type G signals, this same sum delayed by a time T1 and the inverted signal G is delayed by time T2. In this way the rising edge of the signal H is delayed by T1 with respect to the rising edge of one of the inputs of type G and T2 with respect to the falling edge of the signal G of the neuron output.

FIG. 10B shows an exemplary embodiment of a circuit for generating the learning signals A1 and A2.

The signals A1 and A2 are obtained by comparison of the integrated value VA of the signal G with respect to time, with three threshold values: VTH1, VTH2, VTH3. For example, this integrated value can be supplied by a gain amplifier K provided with a constant of time T3, whose transfer function is equal to: K * (l + pT3) p being the Laplace variable, and K being greater than 1. The signal VA is then connected to three comparators referenced respectively to VTH1, VTH2 , VTH3. The logic signals VI, V2, V3 provided by these comparators define four domains for VA, and allow, thanks to a simple coding, the generation of the logic signals Al and A2: Al ≈ V2

A2 = VI.V3 / it is also possible, and advantageous in certain cases, to carry out an entirely logical processing on the signal G to obtain a quantity representative of its integrated value, and to deduce therefrom the value of the couple Al, A2 .

An initialization signal "INIT" is used to reset all type N synapses to the valid state and forces the couple of signals A1 and A2 to zero. Figure 11 shows the interconnection of multiple N neurons and one G neuron, as described above, to form a mutually exclusive domain. The number of elements in a domain is the number of N-type neurons it has. Figure 12 symbolically represents an area of mutual exclusion of three elements.

FIG. 13 A represents the basic principle of the connection of two domains to form a network of mutual exclusion: each neuron N emits a modifiable inhibitory synapse towards each of the neurons N of the opposite domain, and each neuron G emits a non inhibitory synapse modifiable to each of the neurons N and to the neuron G of the opposing domain.

FIG. 13B describes another embodiment of the inhibition interconnections of the G neurons of the external domains on the neurons N of a domain considered. In this embodiment, the individual inhibition of the neurons N of a domain by the neuron G of the opposing domain is replaced by a global inhibition effected by the signal H generated by the neuron G of their own domain. This signal H represents the logical sum of the activities of all G neurons in the opposing domains. The advantage of using this intermediate signal is also to allow temporal control at this level (T1, T2): T1 gives priority to the emergence of neurons N over the action of neurons G, whereas T2 allows propagation of constraints in a network.

Figure 14 is a symbolic representation of a mutual exclusion network made up of two areas.

The input synapses of a neuron allow it to be informed about the state of the surrounding neurons, which constitute its field of reception. By its output, it can modify the state of other neurons, which form its field of action. The reception field of a type G neuron consists only of type N neurons from its own domain and type G neurons from other domains of the network. Its field of action covers all the neurons of the other connected domains, whether directly for type G neurons, or indirectly for type N neurons (via the H signal from each of the other G neurons).

The processing performed by each neuron corresponds to the NOR logical function. Consequently, all the links between the neurons are inhibi¬ tion links, which gives rise to competition between the interconnected neurons (mutual exclusion). Within a domain, there cannot be more than one active neuron. The emergence of a neuron is defined as the passage of this neuron from the inactive state to the active state. When in a domain all type N neurons are inactive, we say that the domain is dissatisfied. The domain becomes satisfied when an N-type neuron emerges.

A type G neuron is active if its domain is unsatisfied and if none of the type G neurons located in its reception field are active. When, in a domain, competition is opened between several neurons from the inactive state, the domain is in a metastable state which is resolved by the emergence of a single neuron. Since the neurons are all almost identical by construction, it is impossible a priori to predict the victorious neuron. However, within a domain, the emergence of a particular neuron can be obtained by the inhibition of all the other neurons in the domain.

Consider a mutually exclusive domain, and a neuron isolated outside of this domain. The field of action of this neuron is made up of all the N-type neurons in the domain, but its state depends only on external conditions. We can establish an association between this isolated neuron and a particular neuron of the domain by invalidating the synapse which connects the second to the first. The activation of the external neuron causes the inhibition of all the other neurons in the domain and allows the emergence of the particular neuron.

The action, on a domain of mutual exclusion, of a neuron outside this domain, is analogous to a selection. Such a selection is said to be individual when it inhibits all the elements of the domain, except one. It is a group selection when it allows competition between several neurons. If all emergence is prohibited, there is total inhibition of the domain. A domain of mutual exclusion, at a given instant, is restricted, if it is subjected to the action of external neurons of which at least one is active at this instant. The domain is said to be free otherwise. Let us consider two domains of mutual exclusion Dl and D2 connected to each other (cf. fig. 13). Before any learning, and in the absence of external constraints, the assembly thus formed does not have a stable state and is the seat of coupled oscillations as shown in the timing diagram of FIG. 15.

Indeed, if an NX neuron emerges in the D1 domain, Gl is inhibited, as well as all the N neurons of D2. There is then the emergence of G2, which inhibits the entire Dl domain via Hl. A neuron NY of D2 can then emerge, which causes the inhibition of G2 and all the neurons N of Dl, then the emergence of Gl and the inhibition of the entire domain D2 via H2, and so right now...

One can establish an association between the NX element of D1 and the NY element of D2 by invalidating the synapses which connect them mutually. In this way, the emergence of NX causes the inhibition of Gl and all the elements of D2 except NY. We thus arrive at a stable state where the only active neurons are NX and NY. Multiple associations can thus be established between elements of D1 and D2. The same element of a domain can be associated with several elements of another domain.

Associative recall is obtained by causing the emergence of a particular element of a domain (by the action of an external neuron), and by leaving the other domain free. The associative mechanism then causes in this area the emergence of an element associated with the first. There are two learning modes which correspond to two types of material achievements:

- supervised learning: the network is made up of neurons N and neurons G not including the circuits used for learning. In a first version, the connection network can be defined in simulation, then produced physically with permanent synapses constituted by diodes. Another possibility is to build a network whose synapses are switching elements controlled by an external system. In both cases, learning is super¬ targeted;

- unsupervised learning; it is carried out directly in the network, consisting of neurons equipped with all the learning circuits described above, that is to say comprising modifiable synapses. The learning takes place automatically on presentation of configurations corresponding to "external signal configurations". We will first describe the implementation of associative memory, and then the operation of unsupervised learning.

The implementation of the associative memory is described here starting from a deliberately very simple example.

Consider two areas:

Dl ≈ (Ader, Blériot, Lilienthal, Montgolfier, Wright)

D2 = (Clément, Etienne, Louis, Joseph, Orville, Otto, Wilbur)

The corresponding mutual exclusion network includes as many N-type neurons as there are elements (i.e. 12), and as many G-type neurons as there are domains (i.e. 2). The connections between all these neurons are represented by the matrix in Figure 16, where a white box corresponds to a valid synapse, and a cross to an invalid synapse. In this matrix, the inputs of each neuron are on the same line, and the outputs of each neuron on the same column. The inhibitory action of each G neuron on the neurons of the opposing domain is represented directly without including the H signals. The two domains are established and interconnected, but the network is still free of all knowledge and does not have any 'stable states. It must be given the information necessary for a coherent database to be established, here in the form of pairs of elements each chosen in a field. In the present case, the following associations can be introduced: Joseph - Montgolfier (1)

Etienne- Montgolfier (2) Otto - Lilienthal (3) Clément- Ader (4) Wilbur - Wright (5) Louis - Blériot (6)

The taking into account of this information corresponds, in the matrix of connections, to the invalidation of the synapses which link together the elements of the same association. The network then has as many stable states as there are registered associations. FIG. 17 indicates, for each invalidated synapse, the number of the associated information.

It now remains to set up the inputs necessary to delimit the field of evolution of the system. The El entry will select "Ader" in the Dl area, the E2 entry will select "Blériot", and so on until E12 for "Wilbur" in D2. Synapses invalidated by this operation are marked "7" in figure 17. We can also prohibit the emergence of certain elements, for example "Joseph" with E13, "Etienne" with E14, "Orville" with E15 and "Wilbur" with El6 (synapses marked "8" in Figure 17).

The mutual exclusion network thus programmed is ready to operate in associative memory. Interrogation consists in restricting one or both domains, and letting the network find for itself a stable state. The questions asked of the network are of the type: "what does this situation mean to you?".

The answers are of two kinds:

- if there is no registered association compatible with all the entry constraints, the network goes into an unstable state, oscillating between several situations satisfying each constraint in turn; if there are one or more associations compatible with the constraints, the network stabilizes on one of them. The others can be obtained by successive eliminations. When all the possible associations have been eliminated, the network goes into an unstable state, identical to the previous case.

For example ; "Lilienthal" question? (activation E3) "Otto" response. (emergence N3, NI1) "Montgolfier" question? (activation E4) "Joseph" response. (emergence N4, N9) question "other than Joseph"? (activation E4, E13) "Etienne" response. (emergence N4, N7) question "other than Joseph and Etienne"? (activation E4, E13, E14) response "no response known". (unstable state) "Wilbur" question? (activation E12) "Wright" response. (emergence NI2, N5) question "Wright other than

Wilbur "? (Activation E5, E16) - response" no response known ". (Unstable state) The association "Orville-Wright", still unknown to the network, must be registered by invalidating the synapses connecting N5 and N10:

- question: "Wright other than Wilbur"? (activation E5, E16)

- answer: "Orville". (emergence N5, N10)

Such a memory can be produced from a neural network comprising interruptible synapses, that is to say synapses which can be irreversibly and permanently invalidated. The realization of the connection matrices described above is carried out by cutting the synapses to be invalidated.

Interruptible synapses can for example consist of fuse connections, or switching elements controlled by flip-flops or memory cells.

Figure 18 illustrates the regime of stress propagation in a mutual exclusion network. This figure represents a linear network comprising three domains A, B, C. Let us call respectively "Ga", "Gb" and "Gc" the neurons "G" of these three domains, and suppose that this network has learned the configurations "Al, Bl, Cl "and" A2, B2, C2 ". In the absence of any constraint, it stabilized for example on "Al, Bl, Cl". Let an external constraint inhibit Al. In domain A, neuron A2 cannot emerge, because of Bl. The neuron "Ga" is therefore activated, and inhibits domain B. A2 can therefore emerge, and "Ga" s 'off. No neuron from domain B can emerge, since Bl is blocked by A2, and B2 is blocked by Cl. The neuron "Gb" therefore emerges, inhibiting domain C; domain A, on the other hand, is protected from this action by blocking the signal "H" for a time T2 after the falling edge of Ga. Since the neuron Cl is no longer active, B2 can emerge in agreement with A2. Gb is deactivated and C2 can emerge, in agreement with B2. We has thus spread a constraint in the network which only affected one area. The new state of the network again satisfies all of the constraints.

This propagation principle also applies to tree networks, thus making it possible to link many domains into complex networks (Figure 19). The choice of time T2 is made taking into account the depth at which it is necessary to propagate the stresses. Figures 20 and 21 illustrate the unsupervised learning process intended to achieve an associative random access memory. Unsupervised learning automatically modifies synapses, in response to variations in external signals and changes in other areas of the network. The learning mechanism is internal to each domain, and acts each time on the synapses of a single neuron in the domain. It ensures, for all configurations, the stable emergence of a neuron "N" in each of the network domains.

There are two kinds of unsupervised learning, depending on the source of the signals acting on the domain: signals from the outside world on the one hand, and states of neurons in other domains on the other. The same mechanism is used in both cases, but the function performed is different: type 1 learning performs a recognition function, while type 2 learning performs the association function. Type 1 learning (see Figure 20)

Type 1 learning takes place when a domain is totally inhibited by one or more signals outside the field of action of the neuron "G" (signals from the "outside world"). All the "N" neurons in the domain are therefore inhibited. The neuron "G" is activated, and the signal "Va" follows an increasing progression, causing the generation of the ascending sequence of the signals "A1, A2" described above, until the configuration "1.0" (FIG. 20). The circuit for controlling the learning of each virgin neuron then generates the command "C /" intended for all of its modifiable synapses, which temporarily become invalid. All the virgin neurons in the domain compete, but only one emerges. For this neuron (NI), the signal "V" becomes inactive, as well as the signal "Z /". The neuron "G" comes back inactive, and the pair of signals "A1, A2" initiates a descending sequence. When its configuration changes to "1,1", the command "C /" becomes inactive again for all virgin neurons (see N2), whose modifiable synapses resume their previous state, since the rising edge of "C /" occurs while the "Z /" signal is still active. On the contrary, the command "C /" is maintained for the neuron which has emerged, until the return to the configuration "0.0" for "Al, A2". The rising front of "C /" occurring while "Z /" is inactive, the neuron definitively places in the "invalid" state those of its modifiable synapses which received an afferent signal active at this time, the others becoming again valid. The type 1 training sequence is then completed.

The present configuration of external signals has just been recorded by the neuron (NI), and by it alone. This neuron can emerge again each time the same configuration of active inputs, even incomplete, presents itself. Type 1 learning occurs only once for each "N" neuron.

When the external configuration changes at the same time for several areas of mutual exclusion interconnected in a network, learning takes place from sequentially, one domain after another, because the "G" neurons inhibit each other. Type 2 learning (see Figure 21)

Type 2 learning takes place when the associative mechanism, in a particular configuration of inputs, does not find an adequate association between two domains connected in a network. In each of these areas, a neuron previously recorded the configuration of the external signals by type 1 learning, and must now associate this configuration with the activity of a neuron from the opposing domain. As long as this association, which corresponds to the interruption of the links between the neurons concerned, is not achieved, the network is the seat of an oscillation, according to the process described above (see Figure 15).

Let NX and NY respectively be the neurons of the domains Dl and D2 having recorded the respective configurations of the external signals arriving on these domains, by a type 1 learning process. An oscillation is established in which Gl and NY emerge in turn. , G2 and NX on the other hand.

In each domain, the pair of signals "A1, A2" generated by the neuron "G" describes an ascending sequence, due to the oscillation of the neuron "G" (see FIG. 21). In its progression, this sequence arrives at the "1,1" configuration. Suppose this happens first in the domain Dl. When the neuron NX, in the normal course of its oscillation, passes to the active state, its learning command is then engaged, and all its synapses still valid pass into the provisionally invalid state. In particular, the synapse that comes from NY is invalidated, the neuron NX is therefore no longer inhibited by NY. In contrast, NY is still inhibited by NX. The oscillation between the two domains therefore continues in another mode, described by the chronogram of the FIG. 21, in which the neuron Gl remains in the inactive state, since learning is already carried out in its domain. The neuron NX returns NY to "0", so G2 goes to "1" and returns NX to "0". The neuron NY can then emerge and reset G2 to "0", which authorizes the emergence of NX which returns NY to "0", and so on. The continuation of the oscillation of G2 allows the sequence of signals "A1, A2" in the domain D2 to continue its ascending progression, while in the domain Dl these signals have already started their descending sequence. This oscillation continues until the same process takes place in the D2 domain, that is to say that the apprenticeship command from NY goes to the active state. All the synapses of NY still valid temporarily go into the invalid state. The neuron NY then remains active, like the neuron NX, the neurons Gl and G2 being both in the inactive state.

Type 2 learning is complete when the learning commands for neurons NX and NY are returned to the inactive state. The synapses which connect NX and NY are therefore definitively in the disabled state. A symmetrical cut was then made between NX and NY, and thus recorded the association between the configurations of external signals recognized respectively by NX and NY in the domains Dl and D2. The network is in a stable state, both NX and NY being active. Subsequently, when the domain D1 again recognizes the configuration recorded by the neuron NX, the emergence of NY will take place in the domain D2 if the latter is left free of constraints. This emergence will correspond to the "evocation" of the configuration recorded by NY. Type 2 learning can be done several times for each neuron "N".

Thanks to the particularly simple structure of the neural network according to the invention, it is possible to create associative memories whose operation is reliable; the learning procedure is easy to carry out and it is therefore possible to envisage multiple applications for the associative memories in accordance with the present invention.

The above description has been provided by way of illustrative and in no way limitative examples and it is obvious that modifications or variations can be made thereto without departing from the scope of the present invention.

In particular, the examples described are cable networks, but it is also possible to make provision for the neural networks according to the invention at least partially in the form of programs.

Claims

CLAIMS 1. Neural network made up of neurons comprising input synapses connected to a summation operator, said neurons being interconnected, characterized in that: - it is made up of fully binary neurons; - the summation operator performs a "NON-OR" logic function such that its output is only active if all of its inputs are at the inactive level; - the neurons are grouped into areas of mutual exclusion which include several neurons of a first type (N) and a single neuron of a second type (G); - in each area of mutual exclusion, the synapses of the neuron of the second type (G) respectively receive the output signals of the neurons of the first type (N) of the domain considered and the output signals of the neurons of the second type (G) of the domains interconnected with the domain considered; and - in each mutual exclusion domain, the synapses of the neurons of the first type (N) respectively receive the output signals of the neurons of the first type (N) of the domain considered, the output signals of the neurons of the first type (N) domains interconnected to the domain considered, at least one inhibition signal coming from neurons of the second type (G) of the domains interconnected to the domain considered and external signals. 2. Neural network according to claim 1, characterized in that the neurons of type (N) are connected by a synapse to the output of each of the neurons (G) of the domains interconnected to their domain. 3. Neural network according to claim 1, characterized in that: each neuron of the second type (G) comprises a circuit generating the inhibition signal (H), said circuit being controlled by the output signal of the type neuron (G) considered and by the output signals of the neurons of the type ( G) domains interconnected with the domain considered; and - the inhibition signal (H) is transmitted to each neuron of type (N) by a synapse connected to the output of the circuit generating the inhibition signal (H) of the neuron of type (G) of the domain considered. 4. Neural network according to claim 3, characterized in that: each neuron of the second type (G) comprises a circuit generating a learning command (A1, A2) controlled by the output signal of this neuron; - neurons of the first type (N) comprise interruptible synapses provided with an associated control circuit and a circuit generating a blocking signal (Z) and a learning signal (C), receiving the command d learning (A1, A2) provided by the neuron of the second type of the domain considered, and an initialization signal (INIT); - in each neuron of the first type (N), the aforementioned blocking (Z) and learning (C) signals are sent to all the interruptible synapses control circuits. 5. Neural network according to claim 4, characterized in that the learning command is a two-bit reflected binary code. 6. Neural network according to claim 4, characterized in that in each neuron of the first type (N), the generator of the training signal (C) receives the initialization signal (INI) and provides at output a blocking signal (Z) which is sent to all interruptible synapses. 7. A method of producing an associative memory constituted by a neural network according to any one of claims 1 to 3, characterized in that a network of neurons connected by permanent synapses is produced according to a determined configuration. 8. A method of producing an associative random access memory constituted by a neural network according to any one of claims 4 to 6, characterized in that, by action of the initialization signal, all the interruptible synapses are set to the state valid and that unsupervised learning is then carried out. 9. A method of producing an associative memory constituted by a neural network according to any one of claims 1 to 3, characterized in that one realizes a network comprising interruptible synapses and in that one performs supervised learning by blocking specific synapses of said network. 10. The method of claim 8 or 9, charac- terized in that each interruptible synapse is consti¬ killed by a fuse. 11. Method according to any one of Claims 8 or 9, characterized in that each interruptible synapse is a switching element controlled by a rocker. 12. Method according to any one of claims 8 or 9, characterized in that each interruptible synapse is a switching element controlled by a memory cell.