TWI903748B - System and method to generate digital output from vector-by-matrix multiplication array - Google Patents

Abstract

In one example, a system comprises a vector-by-matrix multiplication array comprising an array of non-volatile memory cells arranged in rows and columns; and a sigma-delta analog-to-digital converter to receive a current from a column of the vector-by-matrix multiplication array and to generate a digital output in response to the current.

Description

System and method for generating digital output from a vector matrix multiplication array.

[Priority Claim]

This application claims priority to U.S. Provisional Patent Application No. 63/601,049, filed November 20, 2023, entitled "Output Circuit for a Vector-by-Matrix Multiplication Array," and U.S. Patent Application No. 18/426,071, filed January 29, 2024, entitled "Sigma-Delta Analog-to-Digital Converter to Generate Digital Output from Vector-by-Matrix Multiplication Array."

Numerous examples of Σ-Δ analog-to-digital converters and related methods for generating digital outputs from vector-matrix multiplication arrays are revealed.

Artificial neural networks simulate biological neural networks (the central nervous system of animals, especially the brain) and are used to estimate or assess functions that depend on large amounts of input and are often unknown. Artificial neural networks typically consist of layers of interconnected "neurons" that exchange information with each other.

Figure 1 illustrates an artificial neural network, where circles represent neuronal inputs or layers. Connections (called synapses) are indicated by arrows and have numerical weights that can be tuned based on experience. This allows the neural network to adapt to inputs and learn. Typically, a neural network consists of multiple input layers. There are usually one or more intermediate neuron layers and output neuron layers that provide the network's outputs. Neurons at each layer make decisions individually or collectively based on data received from synapses.

One of the major challenges in developing artificial neural networks for high-performance information processing is the lack of sufficient hardware technology. Real neural networks rely on a vast number of synapses to achieve high connectivity between neurons, i.e., extremely high computational parallelism. In principle, this complexity can be achieved using digital supercomputers or dedicated clusters of graphics processing units. However, in addition to high cost, these methods are also hampered by moderate energy efficiency compared to biological networks, primarily because biological networks perform low-precision analog operations and therefore consume far less energy. CMOS analog circuits have been used in artificial neural networks, but given the large number of neurons and synapses, most CMOS implementations have excessively large synapses.

The applicant previously disclosed an artificial (analog) neural network utilizing one or more nonvolatile memory arrays as synapses in U.S. Patent Application Publication 2017/0337466A1, which is incorporated herein by reference. The nonvolatile memory arrays operate as analog neural memory and include nonvolatile memory cells arranged in columns and rows. The neural network includes: a first plurality of synapses wired to receive a first plurality of inputs and generate a first plurality of outputs therefrom; and a first plurality of neurons wired to receive the first plurality of outputs. The first plurality of synapses include a plurality of memory cells, each of which includes: spaced-apart source and drain regions formed in a semiconductor substrate, wherein a channel region extends between the source and drain regions; a floating gate disposed above and insulated from a first portion of the channel region; and a non-floating gate disposed above and insulated from a second portion of the channel region. Each of the plurality of memory cells stores a weighted value corresponding to the number of electrons on the floating gate. The plurality of memory cells multiply the first plurality of inputs by the stored weighted values to generate the first plurality of outputs.

Non-volatile memory cells

Non-volatile memory is well known. For example, U.S. Patent 5,029,130 (“130 Patent”), incorporated herein by reference, discloses a discrete gate array of non-volatile memory cells, which is a type of flash memory cell. This memory cell 210 is shown in FIG. 2. Each memory cell 210 includes a source region 14 and a drain region 16 formed in a semiconductor substrate 12, wherein a channel region 18 is located between the source region and the drain region. A floating gate 20 is formed above and insulated from (and controls the conductivity of) a first portion of the channel region 18, and is formed above a portion of the source region 14. The character line terminal 22 (which is typically coupled to a character line) has a first portion disposed above and insulated from (and controlling the conductivity of) a second portion of the channel region 18, and a second portion extending above and on the floating gate 20. The floating gate 20 and the character line terminal 22 are insulated from the substrate 12 by a gate oxide. The bit line 24 is coupled to the drain region 16.

Memory cell 210 is erased by applying a high positive voltage to word line terminal 22 (whereby electrons are removed from the floating gate). This causes electrons on the floating gate 20 to tunnel through the intermediate insulation from the floating gate 20 to the word line terminal 22 via Fowler-Nordheim (FN) tunneling.

Memory cell 210 is programmed by source-side injection (SSI) of hot electrons (where electrons are placed on a floating gate) by applying a positive voltage to word line terminals 22 and a positive voltage to source region 14. Electron current flows from drain region 16 towards source region 14. When electrons reach the gap between word line terminals 22 and floating gate 20, they are accelerated and heated. Some of the heated electrons are injected into floating gate 20 through the gate oxide due to the attractive electrostatic force from floating gate 20.

Memory cell 210 reads data by applying a positive read voltage to the drain region 16 and word line terminal 22 (connecting the portion of channel region 18 located below the word line terminal). If the floating gate 20 is positively charged (i.e., electrons are erased), the portion of channel region 18 located below the floating gate 20 is also connected, and current flows across channel region 18; this is detected as erased or a "1" state. If the floating gate 20 is negatively charged (i.e., electronically programmed), the portion of channel region located below the floating gate 20 is mostly or completely disconnected, and current does not flow across channel region 18 (or a very small amount of current flows across the channel region); this is detected as programmed or a "0" state.

Table 1 depicts the typical voltage and current ranges that can be applied to the terminals of memory cell 210 for performing read, erase, and programming operations:

Other isolated gate memory cell configurations are known, which are other types of flash memory cells. For example, Figure 3 depicts a four-gate memory cell 310, which includes a source region 14, a drain region 16, a floating gate 20 located above a first portion of a channel region 18, a selection gate 22 located above a second portion of the channel region 18 (typically coupled to a word line WL), a control gate 28 located above the floating gate 20, and an erase gate 30 located above the source region 14. This configuration is described in U.S. Patent 6,747,310, which is incorporated herein by reference for all purposes. Here, except for the floating gate 20, all gates are non-floating gates, meaning they are electrically connected or can be electrically connected to a voltage source. Programming is performed by injecting heated electrons from channel region 18 into the floating gate 20. Erasure is performed by electrons tunneling from the floating gate 20 to the erase gate 30.

Table 2 depicts the typical voltage and current ranges that can be applied to the terminals of memory cell 310 for performing read, erase, and programming operations:

Figure 4 depicts a three-gate memory cell 410, another type of flash memory cell. Memory cell 410 is identical to memory cell 310 in Figure 3, except that it does not have a separate control gate. Erasure operations (where erasure is performed using an erase gate) and read operations are similar to those in Figure 3, except that no control gate bias is applied. Programming operations are also performed without a control gate bias, and therefore, a higher voltage is applied to the source line during programming operations to compensate for the lack of a control gate bias.

Table 3 depicts the typical voltage and current ranges that can be applied to the terminals of memory cell 410 for performing read, erase, and programming operations:

Figure 5 depicts a stacked gate memory cell 510, which is another type of flash memory cell. Memory cell 510 is similar to memory cell 210 in Figure 2, except that a floating gate 20 extends over the entire channel area 18, and a control gate 22 (which will be coupled to the character line here) extends over the floating gate 20. The control gate and the floating gate are separated by an insulating layer (not shown). The erasure is performed by electron tunneling from the source region (FG) to the substrate via the source region (FN). The process is programmed by hot channel electron (CHE) injection in the region between the channel 18 and the drain region 16, with electrons flowing from the source region 14 towards the drain region 16. The read operation is similar to that for a memory cell 210 with a higher control gate voltage.

Table 4 depicts the typical voltage range that can be applied to the terminals and substrate 12 of the memory cell 510 for performing read, erase, and programming operations:

The methods and techniques described herein can be applied to other non-volatile memory technologies, such as, but not limited to, FinFET gate flash or stacked gate flash memory, NAND flash, SONOS (silicon-oxide-nitride-oxide-silicon, charge trapping in nitrides), MONOS (metal-oxide-nitride-oxide-silicon, metal charge trapping in nitrides), ReRAM (resistive RAM), PCM (phase change memory), MRAM (magnetic RAM), FeRAM (ferroelectric RAM), CT (charge trapping) memory, CN (carbon nanotube) memory, OTP (two-level or multi-level one-time programmable), and CeRAM (correlated electronic RAM).

To utilize memory arrays containing one of the nonvolatile memory cell types described above in artificial neural networks, two modifications are made. First, the lines are configured such that each memory cell can be individually programmed, erased, and read without adversely affecting the memory state of other memory cells in the array, as explained further below. Second, continuous (analogous) programming of memory cells is provided.

Specifically, the memory state (i.e., the charge on the floating gate) of each memory cell in the array can independently and continuously change from a completely erased state to a fully programmed state, and vice versa, with minimal interference to other memory cells. This means that the cell memory can effectively analog to or at least store one of many discrete values (e.g., 16 or 64 different values), allowing for extremely precise and individual tuning of all memory cells in the memory array, and making the memory array ideal for storing and fine-tuning the synaptic weights of neural networks.

Neural networks employing non-volatile memory cell arrays

Figure 6 conceptually illustrates a non-limiting example of a neural network utilizing a non-volatile memory array, as described in this invention. This example uses a non-volatile memory array neural network for facial recognition applications, but any other suitable application can be implemented using a neural network based on a non-volatile memory array.

S0 is the input layer. In this example, the input layer is a 32×32 pixel RGB image with 5-bit precision (i.e., three 32×32 pixel arrays, one array for each color (R, G, and B), and each pixel with 5-bit precision). The synapse CB1 that enters layer C1 from input layer S0 applies different weight sets in some cases and shared weights in others. The input image is scanned with a 3×3 pixel overlay filter (core), shifting the filter by 1 pixel (or more than 1 pixel, as specified by the model). Specifically, the values of nine pixels in a 3×3 portion of the image (i.e., the filter or core) are provided to synapse CB1. These nine input values are multiplied by appropriate weights, and after summing the outputs of the multiplications, a single output value is determined and provided by the first synapse of CB1 to generate a pixel in the feature map of layer C1. The 3×3 filter is then shifted one pixel to the right within the input layer S0 (i.e., a row of three pixels is added on the right and a row of three pixels is discarded on the left), thereby providing the nine pixel values in this newly positioned filter to synapse CB1. These pixel values are multiplied by the same weights, and a second single output value is determined by the associated synapse. This process continues until the 3×3 filter has scanned the entire 32×32 pixel image across all three colors and all bits (precision values) of the input layer S0. The process is then repeated using different weight sets to generate different feature maps for layer C1, until all feature maps for layer C1 have been calculated.

In this embodiment of the invention, there are 16 feature maps in layer C1, each with 30×30 pixels. Each pixel is a new feature pixel extracted by multiplying the input by the kernel, and therefore each feature map is a two-dimensional array. Thus, in this embodiment, layer C1 constitutes 16 layers of two-dimensional arrays (note that the terms "layer" and "array" as used herein are logical relations and may not correspond to physical relations; that is, the array may not be oriented in a physical two-dimensional array). Each of the 16 feature maps in layer C1 is generated from one of sixteen different sets of synaptic weights applied to filter scanning. The C1 feature maps can all be different forms of the same image feature, such as boundary recognition. For example, a first image (generated using a first set of weights, shared by all scans used to generate this first image) can identify circular edges, while a second image (generated using a second set of weights different from the first set) can identify rectangular edges, or aspect ratios of certain features, etc.

Before entering layer S1 from layer C1, an excitation function P1 (pooling) is applied, with pooling derived from the values of consecutive, non-overlapping 2×2 regions in each feature map. The purpose of pooling function P1 is to average the values of nearby locations (or, alternatively, to use a max function), for example, to reduce the dependency of edge locations and decrease the data size before moving to the next stage. At layer S1, there are 16 15×15 feature maps (i.e., sixteen different arrays, each with 15×15 pixels). The synapse CB2, which allows entry from layer S1 into layer C2, scans the map in layer S1 using a 4×4 filter, with each filter shifted by one pixel. At layer C2, there are 22 12×12 feature maps. An excitation function P2 (pooling) is applied before entering layer S2 from layer C2. This pooling is derived from the values of consecutive, non-overlapping 2×2 regions in each feature map. At layer S2, there are 22 6×6 feature maps. An excitation function (pooling) is applied at the synapse CB3, which connects layer S2 to layer C3. Each neuron in layer C3 is connected to each feature map in layer S2 via individual synapses in CB3. At layer C3, there are 64 neurons. The synapse CB4, which connects layer C3 to the output layer S3, completely connects C3 to S3; that is, each neuron in layer C3 is connected to every neuron in layer S3. The output at layer S3 comprises 10 neurons, with the highest output neuron determining the category. This output can, for example, indicate the identification or classification of the content of the original image.

Each synaptic layer is implemented using arrays or portions of nonvolatile memory cells.

Figure 7 is a block diagram of the array that can be used for that purpose. The Vector Matrix Multiplication (VMM) array 32 includes non-volatile memory cells and is used as synapses between layers (such as CB1, CB2, CB3, and CB4 in Figure 6). Specifically, the VMM array 32 includes a non-volatile memory cell array 33, an erase gate and character line gate decoder 34, a control gate decoder 35, a bit line decoder 36, and a source line decoder 37, which decode the respective inputs of the non-volatile memory cell array 33. The inputs to the VMM array 32 can come from the erase gate and character line gate decoder 34 or from the control gate decoder 35. In this example, the source line decoder 37 also decodes the output of the non-volatile memory cell array 33. Alternatively, the bit line decoder 36 can decode the output of the non-volatile memory cell array 33.

The non-volatile memory cell array 33 serves two purposes. First, it stores the weights to be used by the VMM array 32. Second, the non-volatile memory cell array 33 efficiently multiplies the input by the weights stored in it and adds the results along the output lines (source lines or bit lines) to produce an output that will serve as the input to the next layer or the final layer. By performing multiplication and addition functions, the non-volatile memory cell array 33 eliminates the need for separate multiplication and addition logic circuits and is also power efficient due to its in-situ memory operations.

The output of the nonvolatile memory cell array 33 is supplied to a differential summer (such as a summing operational amplifier or a summing current mirror) 38, which sums the output of the nonvolatile memory cell array 33 to produce a single value for convolution. The differential summer 38 is configured to perform summation of positive and negative weights.

The total output value of the difference summer 38 is then supplied to the excitation function block 39, which corrects the output. The excitation function block 39 can provide a sigmoid, hyperbolic tangent, or ReLU function. The corrected output value of the excitation function block 39 becomes an element of the feature map of the next layer (e.g., C1 in Figure 6), and is then applied to the next synapse to produce the next feature map layer or the final layer. Therefore, in this example, the nonvolatile memory cell array 33 constitutes a plurality of synapses (which receive inputs from the preceding neuronal layer or from an input layer such as an image database), and the summing operational amplifier 38 and the excitation function block 39 constitute a plurality of neurons.

The inputs (WLx, Egx, CGx, and optionally BLx and SLx) to the VMM array 32 in Figure 7 can be analog, binary, or digital (in which case, the DAC is configured to convert digital bits to the appropriate input analog level), and the outputs can be analog, binary, or digital (in which case, the output ADC is configured to convert the output analog level to digital bits).

Figure 8 is a block diagram depicting the use of the various layers of VMM array 32, labeled VMM arrays 32a, 32b, 32c, 32d, and 32e. As shown in Figure 8, the input to Inputx is converted from digital to analog by the digital-to-analog converter 31 and provided to the input VMM array 32a. The converted analog input can be voltage or current. The first-level input D/A conversion can be performed using a function or LUT (lookup table) that maps the input Inputx to the appropriate analog level for the matrix multiplier of the input VMM array 32a. Input conversion can also be performed using an analog-to-analog (A/A) converter to convert external analog inputs to mapped analog inputs to the input VMM array 32a.

The output generated by the input VMM array 32a is set as the input to the next VMM array (hidden level 1) 32b, which in turn generates an output, which is set as the input to the next VMM array (hidden level 2) 32c, and so on. The various layers of the VMM array 32 serve as different layers of synapses and neurons in the convolutional neural network (CNN). Each VMM array 32a, 32b, 32c, 32d, and 32e can be a separate physical non-volatile memory array, or multiple VMM arrays can utilize different portions of the same physical non-volatile memory array, or multiple VMM arrays can utilize overlapping portions of the same physical non-volatile memory array. The example shown in Figure 8 contains five layers (32a, 32b, 32c, 32d, 32e): one input layer (32a), two hidden layers (32b, 32c), and two fully interconnected layers (32d, 32e). Those familiar with this technique should understand that this is merely an example, and the system may alternatively contain more than two hidden layers and more than two fully linked layers.

Vector Matrix Multiplication (VMM) Array

Figure 9 depicts a neuronal VMM array 900, which is particularly suitable for memory cells 310 as shown in Figure 3, and is used as the synaptic and neuronal portion between the input layer and the next layer. The VMM array 900 includes a memory array 901 of nonvolatile memory cells and a reference array 902 of nonvolatile reference memory cells (at the top of the array). Alternatively, another reference array may be placed at the bottom.

In the VMM array 900, control gate lines such as control gate line 903 extend vertically (therefore, reference array 902 is orthogonal to control gate line 903 in the column direction), and erase gate lines such as erase gate line 904 extend horizontally. Here, the inputs to the VMM array 900 are set on the control gate lines (CG0, CG1, CG2, CG3), and the outputs of the VMM array 900 appear on the source lines (SL0, SL1). In one example, even-numbered columns are used, and in another example, odd-numbered columns are used. The current placed on each source line (SL0, SL1, etc.) is summed against all currents from memory cells connected to that specific source line.

As described in this paper regarding neural networks, the nonvolatile memory cells of VMM array 900, namely memory cells 310 of VMM array 900, can be configured to operate in subcritical regions.

The nonvolatile reference memory cell and the nonvolatile memory cell described in this article are biased in weak inversion (subcritical region): Ids = Io * e ^{(Vg - Vth)/nVt} = w * Io * e ^(Vg)/nVt

Where w = e ^(-Vth)/nVt

Where Ids is the drain-to-source current; Vg is the gate voltage on the memory cell; Vth is the critical voltage of the memory cell; Vt is the thermoelectric pressure = k*T/q, where k is the Boltzmann constant, T is the temperature in Kelvin, and q is the electron charge; n is the slope factor = 1 + (Cdep/Cox), where Cdep = the capacitance of the depletion layer, and Cox is the capacitance of the gate oxide layer; Io is the memory cell current at the gate voltage equal to the critical voltage, and Io is the sum of (Wt/L)*u*Cox*(n-1)*Vt The ratio is ² , where u is the carrier mobility of the memory cell, and Wt and L are the width and length, respectively.

For an I-to-V logarithmic converter that uses memory cells (such as reference memory cells or peripheral memory cells) or transistors to convert input current into input voltage: Vg = n * Vt * log[Ids/wp * Io]

Where wp represents the w in the reference or peripheral memory cell.

For a memory array used as a vector matrix multiplier (VMM) with current input, the output current is: Iout = wa * Io * e ^(Vg)/nVt , or Iout = (wa/wp) * Iin = W * Iin

W=e ^{(Vthp-Vtha)/nVt}

Here, wa = w of each memory cell in the memory array.

Vthp is the effective threshold voltage of the peripheral memory cell, and Vtha is the effective threshold voltage of the main (data) memory cell. It should be noted that the threshold voltage of the transistor is a function of the substrate bias voltage, and the substrate bias voltage, expressed as Vsb, can be modulated to compensate for various conditions at this temperature. The threshold voltage Vth can be expressed as: Vth = Vth0 + γ(SQRT｜Vsb - 2*φF) - SQRT｜2*φF｜)

Where Vth0 is the threshold voltage with zero substrate bias, φF is the surface potential, and γ is the volume effect parameter.

Character lines or control gates can be used as inputs for input voltages in memory cells.

Alternatively, the flash memory cells of the VMM array described in this paper can be configured to operate in a linear region: Ids = β*(Vgs - Vth)*Vds; β = u*Cox*Wt/L.

W = α(Vgs - Vth)

This means that the weight W in the linear region is proportional to (Vgs - Vth).

Character lines, control gates, bit lines, or source lines can be used as inputs to memory cells operating in the linear region. Bit lines or source lines can be used as outputs to memory cells.

For I-to-V linear converters, memory cells (e.g., reference memory cells or peripheral memory cells) or transistors operating in the linear region can be used to linearly convert input/output currents into input/output voltages.

Alternatively, the memory cells of the VMM array described herein can be configured to operate in saturation regions: Ids = ½ * β * (Vgs - Vth) ^² ; β = u * Cox * Wt/L

Wα(Vgs-Vth) ² means that the weight W is proportional to (Vgs-Vth) ² .

Character lines, control gates, or erase gates can be used as inputs to memory cells operating in the saturation region. Bit lines or source lines can be used as outputs to output neurons.

Alternatively, the memory cells of the VMM array described herein can be used in all regions or combinations thereof (subcritical regions, linear regions, or saturation regions) of any or multiple layers of a neural network.

Further examples of the VMM array 32 in Figure 7 are described in U.S. Patent No. 10,748,630, which is incorporated herein by reference. As described in that application, source lines or bit lines can be used as neural outputs (current summation outputs).

Figure 10 depicts a neuronal VMM array 1000, which is particularly suitable for memory cells 210 as shown in Figure 2 and is used as a synapse between the input layer and the next layer. The VMM array 1000 includes a memory array 1003 of nonvolatile memory cells, a reference array 1001 of first nonvolatile reference memory cells, and a reference array 1002 of second nonvolatile reference memory cells. The reference arrays 1001 and 1002, arranged in the row direction of the array, are used to convert current inputs flowing to terminals BLR0, BLR1, BLR2, and BLR3 into voltage inputs to WL0, WL1, WL2, and WL3. In practice, the first and second nonvolatile reference memory cells are diode-connected through-type multiplexers 1014 (partially described), through which current input flows. The reference cells are tuned (e.g., programmed) to a target reference level. The target reference level is provided by a reference miniature array matrix (not shown).

The memory array 1003 serves two purposes. First, its storage will be based on the weights used by the VMM array 1000 in its individual memory cells. Second, the memory array 1003 effectively multiplies the inputs (i.e., the current inputs provided in terminals BLR0, BLR1, BLR2 and BLR3, wherein reference arrays 1001 and 1002 convert such current inputs into input voltages to supply word lines WL0, WL1, WL2 and WL3) by the weights stored in the memory array 1003, and then adds all the results (memory cell currents) to generate outputs on the respective bit lines (BL0 to BLN), which will be the input to the next layer or the input to the final layer. By performing multiplication and addition functions, the memory array 1003 eliminates the need for individual multiplication and addition logic circuits, and is also power efficient. Here, voltage inputs are located on word lines WL0, WL1, WL2, and WL3, and outputs appear on individual bit lines BL0 to BLN during read (inference) operations. The current placed on each of the bit lines BL0 to BLN performs a summation function on the current from all non-volatile memory cells connected to the bit lines.

Table 5 depicts the operating voltages and currents used for the VMM array 1000. The row indicators in the table are set to the following voltages: word lines for selected cells, word lines for unselected cells, bit lines for selected cells, bit lines for unselected cells, source lines for selected cells, and source lines for unselected cells. Column indicators are used for read, erase, and programming operations.

Figure 11 depicts a neuronal VMM array 1100, which is particularly suitable for memory cells 210 as shown in Figure 2, and is used as a synaptic and neuronal portion between the input layer and the next layer. The VMM array 1100 includes a memory array 1103 of nonvolatile memory cells, a reference array 1101 of first nonvolatile reference memory cells, and a reference array 1102 of second nonvolatile reference memory cells. Reference arrays 1101 and 1102 extend in the column direction of the VMM array 1100. The VMM array is similar to VMM 1000, except that in VMM array 1100, the character lines extend in the vertical direction. Here, inputs are set on character lines (WLA0, WLB0, WLA1, WLB1, WLA2, WLB2, WLA3, WLB3), and outputs appear on source lines (SL0, SL1) during read operations. The current placed on each source line is summed against all currents from the memory cells connected to that particular source line.

Table 6 depicts the operating voltages and currents used for the VMM array 1100. The row indicators in the table are set to the following voltages: word lines for selected cells, word lines for unselected cells, bit lines for selected cells, bit lines for unselected cells, source lines for selected cells, and source lines for unselected cells. Column indicators are used for read, erase, and programming operations.

Figure 12 depicts a neuronal VMM array 1200, which is particularly suitable for memory cells 310 as shown in Figure 3, and is used as a synaptic and neuronal portion between the input layer and the next layer. The VMM array 1200 includes a memory array 1203 of nonvolatile memory cells, a reference array 1201 of first nonvolatile reference memory cells, and a reference array 1202 of second nonvolatile reference memory cells. Reference arrays 1201 and 1202 are used to convert current inputs flowing to terminals BLR0, BLR1, BLR2, and BLR3 into voltage inputs CG0, CG1, CG2, and CG3. In practice, the first and second non-volatile reference memory cells are diode-connected through-multiplexers 1212 (partially shown), with current input flowing through BLR0, BLR1, BLR2, and BLR3. Each multiplexer 1212 includes a separate multiplexer 1205 and a series transistor 1204 to ensure a constant voltage on the bit lines (such as BLR0) of each of the first and second non-volatile reference memory cells during read operations. The reference cell is tuned to the target reference level.

The memory array 1203 serves two purposes. First, it stores the weights that will be used by the VMM array 1200. Second, the memory array 1203 effectively multiplies the inputs (current inputs provided to terminals BLR0, BLR1, BLR2, and BLR3, wherein reference arrays 1201 and 1202 convert these current inputs into input voltages to supply to the control gates (CG0, CG1, CG2, and CG3)) by the weights stored in the memory array, and then adds all the results (cell currents) to produce an output that appears on BL0 to BLN and will be the input to the next layer or the final layer. By performing multiplication and addition functions, the memory array eliminates the need for individual multiplication and addition logic circuits, and is also power-efficient. Here, the inputs are located on control gate lines (CG0, CG1, CG2, and CG3), and the outputs appear on bit lines (BL0 to BLN) during read operations. The current placed on each bit line performs a summation function on all currents from the memory cells connected to the bit positioner lines.

The VMM array 1200 performs unidirectional modulation on the non-volatile memory cells in the memory array 1203. That is, each non-volatile memory cell is erased and then partially programmed until the desired charge is reached at the floating gate. If excessive charge is placed at the floating gate (causing an error value to be stored in the cell), the cell is erased, and the sequence of partial programming operations restarts. As shown, two columns sharing the same erase gate line (such as EG0 or EG1) are erased together (this can be called a page erase), and then each cell is partially programmed until the desired charge is reached at the floating gate.

Table 7 depicts the operating voltages and currents used in the VMM array 1200. The row indicators in this table are set to the following voltages: word lines for selected cells, word lines for unselected cells, bit lines for selected cells, bit lines for unselected cells, control gate for selected cells, control gate for unselected cells in the same sector as the selected cell, control gate for unselected cells in a different sector than the selected cell, erase gate for selected cells, erase gate for unselected cells, source lines for selected cells, and source lines for unselected cells. The column indicators handle read, erase, and programmed operations.

Figure 13 depicts a neuronal VMM array 1300, which is particularly suitable for memory cells 310 as shown in Figure 3, and is used as a synaptic and neuronal portion between the input layer and the next layer. The VMM array 1300 includes a memory array 1303 of nonvolatile memory cells, a reference array 1301 of first nonvolatile reference memory cells, and a reference array 1302 of second nonvolatile reference memory cells. EG lines EGR0, EG0, EG1, and EGR1 extend vertically, while CG lines CG0, CG1, CG2, and CG3 and WL lines WL0, WL1, WL2, and WL3 extend horizontally. VMM array 1300 is similar to VMM array 1400, but the difference is that VMM array 1300 implements bidirectional tuning. Due to the use of individual EG lines, individual cells can be completely erased, partially programmed, or partially erased as needed to achieve the desired charge on the floating gate. As shown, reference arrays 1301 and 1302 convert the input current in terminals BLR0, BLR1, BLR2, and BLR3 into control gate voltages CG0, CG1, CG2, and CG3 to be applied to the memory cells in the column direction (operated by the diode-connected reference cell through multiplexer 1314). The current output (neuron) is located on bit lines BL0 to BLN, where each bit line sums the currents from all non-volatile memory cells connected to the Peter position bit line.

Table 8 depicts the operating voltages and currents used in the VMM array 1300. The row indicators in this table are set to the following voltages: word lines for selected cells, word lines for unselected cells, bit lines for selected cells, bit lines for unselected cells, control gate for selected cells, control gate for unselected cells in the same sector as the selected cell, control gate for unselected cells in a different sector than the selected cell, erase gate for selected cells, erase gate for unselected cells, source lines for selected cells, and source lines for unselected cells. The column indicators handle read, erase, and programmed operations.

Figure 22 depicts a neuronal VMM array 2200, which is particularly suitable for memory cell 210 as shown in Figure 2, and is used as a synapse and neuronal portion between the input layer and the next layer. In the VMM array 2200, inputs INPUT ₀ , ..., INPUT _N are received on bit lines BL ₀ , ..., BL _N , respectively, and outputs OUTPUT ₁ , OUTPUT ₂ , OUTPUT ₃ and OUTPUT ₄ are generated on source lines SL ₀ , SL ₁ , SL ₂ and SL ₃ , respectively.

Figure 23 depicts a neuronal VMM array 2300, which is particularly suitable for the memory cell 210 shown in Figure 2, and is used as a synapse and neuronal portion between the input layer and the next layer. In this example, inputs INPUT ₀ , INPUT ₁ , INPUT ₂ and INPUT ₃ are received on source lines SL ₀ , SL ₁ , SL ₂ and SL ₃ , respectively, and outputs OUTPUT ₀ , ..., OUTPUT _N are generated on bit lines BL ₀ , ..., BL _N.

Figure 24 depicts a neuronal VMM array 2400, which is particularly suitable for memory cell 210 as shown in Figure 2, and is used as a synapse and neuronal portion between the input layer and the next layer. In this example, inputs INPUT ₀ , ..., INPUT _M are received on word lines WL ₀ , ..., WL _M , respectively, and outputs OUTPUT ₀ , ..., OUTPUT _N are generated on bit lines BL ₀ , ..., BL _N.

Figure 25 depicts a neuronal VMM array 2500, which is particularly suitable for memory cell 310 as shown in Figure 3, and is used as a synapse and neuronal portion between the input layer and the next layer. In this example, inputs INPUT ₀ , ..., INPUT _M are received on word lines WL ₀ , ..., WL _M , respectively, and outputs OUTPUT ₀ , ..., OUTPUT _N are generated on bit lines BL ₀ , ..., BL _N.

Figure 26 depicts a neuronal VMM array 2600, which is particularly suitable for memory cell 410 as shown in Figure 4, and is used as a synapse and neuronal portion between the input layer and the next layer. In this example, inputs INPUT ₀ , ..., INPUT _N are received on vertical control gate lines CG ₀ , ..., CG _N , respectively, and outputs OUTPUT ₁ and OUTPUT ₂ are generated on source lines SL ₀ and SL ₁ .

Figure 27 depicts a neuronal VMM array 2700, which is particularly suitable for the memory cell 410 shown in Figure 4, and is used as a synapse and neuronal portion between the input layer and the next layer. In this example, inputs INPUT ₀ , ..., INPUT _N are received on the gates of bit line control gates 2701-1, 2701-2, ..., 2701-(N-1) and 2701-N, respectively, which are coupled to bit lines BL ₀ , ..., BL _N. Example outputs OUTPUT ₁ and OUTPUT ₂ are generated on source lines SL ₀ and SL ₁ .

Figure 28 depicts a neuronal VMM array 2800, which is particularly suitable for memory cells 310 as shown in Figure 3, memory cells 510 as shown in Figure 5, and memory cells 710 as shown in Figure 7, and is used as the synaptic and neuronal portion between the input layer and the next layer. In this example, inputs INPUT ₀ , ..., INPUT _M are received on word lines WL ₀ , ..., WL _M , and outputs OUTPUT ₀ , ..., OUTPUT _N are generated on bit lines BL ₀ , ..., BL _N, respectively.

Figure 29 depicts a neuronal VMM array 2900, which is particularly suitable for memory cells 310 as shown in Figure 3, memory cells 510 as shown in Figure 5, and memory cells 710 as shown in Figure 7, and is used as the synaptic and neuronal portion between the input layer and the next layer. In this example, inputs INPUT ₀ , ..., INPUT _M are received on control gate lines CG ₀ , ..., CG _M. Outputs OUTPUT ₀ , ..., OUTPUT _N are generated on vertical source lines SL ₀ , ..., SL _N , respectively, wherein each source line SL _i is coupled to the source line of all memory cells in row i.

Figure 30 depicts a neuronal VMM array 3000, which is particularly suitable for memory cells 310 as shown in Figure 3, memory cells 510 as shown in Figure 5, and memory cells 710 as shown in Figure 7, and is used as the synaptic and neuronal portion between the input layer and the next layer. In this example, inputs INPUT ₀ , ..., INPUT _M are received on control gate lines CG ₀ , ..., CG _M. Outputs OUTPUT ₀ , ..., OUTPUT _N are generated on vertical bit lines BL ₀ , ..., BL _N , respectively, where each bit line BL _i is coupled to the bit lines of all memory cells in row i.

Long Short-Term Memory

Previous techniques included a concept known as Long Short-Term Memory (LSTM). LSTM cells are commonly used in neural networks. LSTM allows neural networks to remember information at predetermined time intervals and use that information in subsequent operations. A known LSTM cell consists of a cell, an input gate, an output gate, and a forget gate. These three gates regulate the flow of information entering and leaving the cell and the time intervals at which information is remembered in the LSTM. Virtual Memory Models (VMMs) are particularly well-suited for LSTM cells.

Figure 14 illustrates an example LSTM 1400. This example LSTM 1400 includes cells 1401, 1402, 1403, and 1404. Cell 1401 receives the input vector _x0 and produces the output vector _h0 and cell state vector _c0 . Cell 1402 receives the input vector _x1 , the output vector (hidden state) _h0 from cell 1401, and the cell state _c0 from cell 1401, and produces the output vector _h1 and cell state vector _c1 . Cell 1403 receives the input vector _x2 , the output vector (hidden state) _h1 from cell 1402, and the cell state _c1 from cell 1402, and produces the output vector _h2 and cell state vector _c2 . Cell 1404 receives input vector _x3 , output vector (hidden state) _h2 from cell 1403, and cell state _c2 from cell 1403, and generates output vector _h3 . Additional cells can be used, and an LSTM with four cells is just an example.

Figure 15 illustrates an example implementation of LSTM cell 1500, which can be used in cells 1401, 1402, 1403, and 1404 in Figure 14. LSTM cell 1500 receives an input vector x(t), a cell state vector c(t-1) from the previous cell, and an output vector h(t-1) from the previous cell, and generates the cell state vector c(t) and the output vector h(t).

LSTM cell 1500 includes S-shaped function devices 1501, 1502, and 1503, each using numbers between 0 and 1 to control the amount of each component of the input vector allowed to pass to the output vector. LSTM cell 1500 also includes hyperbolic tangent devices 1504 and 1505 for applying a hyperbolic tangent function to the input vector, multiplier devices 1506, 1507, and 1508 for multiplying two vectors together, and adder device 1509 for adding two vectors together. The output vector h(t) can be provided to the next LSTM cell in the system, or accessed for other purposes.

Figure 16 depicts LSTM cell 1600, which is an example of an implementation of LSTM cell 1500. For ease of reading, the same designations used for LSTM cell 1500 are applied to LSTM cell 1600. S-shaped function devices 1501, 1502, and 1503, and hyperbolic tangent device 1504 each contain multiple VMM arrays 1601 and excitation function blocks 1602. Therefore, it can be seen that VMM arrays are particularly suitable for LSTM cells used in certain neural network systems. Multiplier devices 1506, 1507, and 1508, and adder device 1509 are implemented digitally or analogously. Excitation function block 1602 can be implemented digitally or analogously.

Figure 17 illustrates an alternative to LSTM cell 1600 (and another example of an implementation of LSTM cell 1500). In Figure 17, sigmoid function devices 1501, 1502, and 1503 and hyperbolic tangent device 1504 share the same physical hardware (VMM array 1701 and excitation function block 1702) in a time-multiplexed manner. The LSTM cell 1700 also includes: a multiplier 1703 for multiplying two vectors together; an adder 1708 for adding two vectors together; a hyperbolic tangent 1505 (containing the excitation function block 1702); a register 1707 for storing the value i(t) when i(t) is output from the sigmoid function block 1702; a register 1704 for storing the value f(t)*c(t-1) when it is output from the multiplier 1703 via the multiplexer 1710; and a register 1705 for storing the value i(t)*c(t-1) when it is output from the multiplier 1703. u(t) is stored when it is output from multiplier device 1703 via multiplexer 1710; and register 1706 is used to store the value o(t)*c~(t) when it is output from multiplier device 1703 via multiplexer 1710 and multiplexer 1709.

LSTM cell 1600 contains multiple sets of VMM arrays 1601 and individual excitation function blocks 1602, while LSTM cell 1700 contains one set of VMM arrays 1701 and excitation function blocks 1702, which are used to represent multiple layers in an instance of LSTM cell 1700. LSTM cell 1700 will require less space than LSTM 1600 because LSTM cell 1700 requires 1/4 of the space for VMMs and excitation function blocks compared to LSTM cell 1600.

As can be further understood, an LSTM cell will typically contain multiple VMM arrays, each utilizing functionality provided by certain circuit blocks external to the VMM arrays, such as summer and excitation function blocks, and high-voltage generation blocks. Providing individual circuit blocks for each VMM array would require a significant amount of space within the semiconductor device and would be somewhat inefficient. Therefore, the examples described below reduce the circuitry used externally to the VMM arrays themselves.

Gate return unit

Analogous VMM implementations can be used in GRU (Gate Recursive Unit) systems. A GRU is the gate mechanism in a recursive neural network. GRUs are similar to LSTMs, but a GRU cell typically contains fewer components than an LSTM cell.

Figure 18 illustrates an example GRU 1800. This example GRU 1800 includes cells 1801, 1802, 1803, and 1804. Cell 1801 receives input vector _x0 and produces output vector _h0 . Cell 1802 receives input vector _x1 and output vector _h0 from cell 1801, and produces output vector _h1 . Cell 1803 receives input vector _x2 and output vector _h1 (in the hidden state) from cell 1802, and produces output vector _h2 . Cell 1804 receives input vector _x3 and output vector _h2 (in the hidden state) from cell 1803, and produces output vector _h3 . Additional cells can be used, and a GRU with four cells is just one example.

Figure 19 illustrates an example implementation of GRU cell 1900, which can be used in cells 1801, 1802, 1803, and 1804 of Figure 18. GRU cell 1900 receives an input vector x(t) and an output vector h(t-1) from the previous GRU cell, and generates an output vector h(t). GRU cell 1900 includes S-shaped function devices 1901 and 1902, each of which applies numbers between 0 and 1 to the components of the output vector h(t-1) and the input vector x(t). The GRU cell 1900 also includes a hyperbolic tangent device 1903 for applying the hyperbolic tangent function to the input vector, multiple multiplier devices 1904, 1905, and 1906 for multiplying two vectors together, an adder device 1907 for adding two vectors together, and a complementary device 1908 for subtracting the input from 1 to produce the output.

Figure 20 depicts GRU cell 2000, which is an example of an implementation of GRU cell 1900. For ease of reading, the same designations from GRU cell 1900 are used in GRU cell 2000. As can be seen in Figure 20, the sigmoid function devices 1901 and 1902 and the hyperbolic tangent device 1903 each contain multiple VMM arrays 2001 and excitation function blocks 2002. Therefore, it can be seen that VMM arrays are particularly used in GRU cells used in certain neural network systems. Multiplier devices 1904, 1905, and 1906, adder device 1907, and complement device 1908 are implemented digitally or analogously. Excitation function block 2002 can be implemented digitally or analogously.

Figure 21 illustrates an alternative to GRU cell 2000 (and another example of an implementation of GRU cell 1900). In Figure 21, GRU cell 2100 utilizes a VMM array 2101 and an excitation function block 2102, which, when configured as a sigmoid function, uses numbers between 0 and 1 to control the amount of each component in the input vector allowed to pass through to the output vector. In Figure 21, the sigmoid function devices 1901 and 1902 and the hyperbolic tangent device 1903 share the same physical hardware (VMM array 2101 and excitation function block 2102) in a time-multiplexed manner. The GRU cell 2100 also includes: a multiplier 2103 for multiplying two vectors together; an adder 2105 for adding two vectors together; a complement 2109 for subtracting the input from 1 to produce the output; a multiplexer 2104; a register 2106 for storing the value h(t-1)*r(t) when it is output from the multiplier 2103 via the multiplexer 2104; and a register 2107 for storing the value h(t-1)*r(t) when it is output from the multiplier 2103. When z(t) is output from multiplier device 2103 via multiplexer 2104, its value is stored; and a register 2108 is used to store the value when h^(t)*(1-z(t)) is output from multiplier device 2103 via multiplexer 2104.

GRU cell 2000 contains multiple sets of VMM array 2001 and excitation function blocks 2002, while GRU cell 2100 contains one set of VMM array 2101 and excitation function blocks 2102, which is used to represent multiple layers in an instance of GRU cell 2100. GRU cell 2100 will require less space than GRU cell 2000 because GRU cell 2100 requires 1/3 of the space for VMM and excitation function blocks.

As can be further understood, a GRU system will typically comprise multiple VMM arrays, each utilizing functionality provided by external circuit blocks, such as summer and excitation function blocks, and high-voltage generation blocks. Providing individual circuit blocks for each VMM array would require significant space within the semiconductor device and would be somewhat inefficient. Therefore, the examples described below reduce the circuitry used externally to the VMM arrays themselves.

The inputs to the VMM array can be analog level, binary level, pulse, time-modulated pulse, or digital bits (in which case, a DAC is used to convert the digital bits to the appropriate input analog level), and the outputs can be analog level, binary level, timing pulse, pulse, or digital bits (in which case, an output ADC is used to convert the output analog level to digital bits).

Generally, for each memory cell in a VMM array, each weight W can be implemented using a single memory cell, a differential cell, or two mixed memory cells (the average of the two cells). In the case of differential cells, two memory cells are used to implement the weight W as a differential weight (W = W+ - W-). In the case of two mixed memory cells, two memory cells are used to implement the weight W as the average of the two cells.

Figure 31 illustrates a VMM system 3100. In some embodiments, the weights W stored in the VMM array are stored as differential pairs W+ (positive weights) and W- (negative weights), where W = (W+) - (W-). In the VMM system 3100, half of the bit lines are designated as W+ lines, i.e., the bit lines connected to the memory cells where the positive weights W+ will be stored, and the other half of the bit lines are designated as W- lines, i.e., the bit lines connected to the memory cells where the negative weights W- will be implemented. The W- lines are interspersed among the W+ lines. Subtraction operations are performed by summing circuits that receive current from the W+ and W- lines, such as summing circuits 3101 and 3102. The outputs of the W+ and W- lines are combined to effectively derive W = W+ - W- for each pair of (W+, W-) cells for all pairs of (W+, W-) lines. Although the alternating arrangement of the W- lines among the W+ lines has been described above, in other examples, the W+ and W- lines can be positioned arbitrarily anywhere in the array.

Figure 32 illustrates another example. In the VMM system 3210, positive weights W+ are implemented in a first array 3211, and negative weights W- are implemented in a second array 3212, which is separate from the first array. The resulting weights are appropriately combined using a summing circuit 3213.

Figure 33 depicts a VMM system 3300. The weights W stored in the VMM array are stored as difference pairs W+ (positive weight) and W- (negative weight), where W = (W+) - (W-). The VMM system 3300 includes arrays 3301 and 3302. Half of the bit lines in each of arrays 3301 and 3302 are designated as W+ lines, i.e., bit lines connected to memory cells where positive weights W+ will be stored, and the other half of the bit lines in each of arrays 3301 and 3302 are designated as W- lines, i.e., bit lines connected to memory cells where negative weights W- are implemented. W- lines are interspersed among the W+ lines in an alternating manner. The subtraction operation is performed by summing circuits that receive current from the W+ and W- lines, such as summing circuits 3303, 3304, 3305, and 3306. The outputs of the W+ and W- lines from each of arrays 3301 and 3302 are combined to effectively derive W = W+ - W- for each pair of (W+, W-) cells of all pairs of (W+, W-) lines. Furthermore, the W values from each array 3301 and 3302 can be further combined using summing circuits 3307 and 3308, so that each W value is the result of subtracting the W value from array 3302 from the W value from array 3301. This means that the final result from summing circuits 3307 and 3308 is one of two difference values.

This analogy is used to describe how non-volatile memory cells in an analogy to those in a neural memory system are erased and programmed to maintain a highly specific and precise amount of charge, i.e., the number of electrons, in a floating gate. For example, each floating gate can retain one of N different values, where N is the number of different weights that can be indicated by each cell. Examples of N include 16, 32, 64, 128, and 256.

Previous technology systems required a considerable area and involved significant latency at the output stage. For example, multiple clock cycles were used to convert the analog current received from the VMM array into digital output data.

To increase the overall operating speed of a system, representing some or all of an artificial neural network, it is necessary to reduce the latency at the output points.

Numerous examples of output circuits and related methods used in neural network arrays are presented.

12: Semiconductor substrate

14: Source Area

16: Jiji Area

18: Passage Area

20: Floating gate pole

22: Character Line Terminal

24: Bitline

28: Control Gate

30: Erasure of the gate pole

31: Digital-to-Analog Converter

32,32a,32b,32c,32d,32e,1601,1701,2001,2101,3301,3302: Vector-Matrix Multiplication Array

33: Non-volatile memory cell arrays

34: Gate and character line eraser decoder

35: Control Gate Decoder

36-bit line decoder

37: Source Line Decoder

38: Differential summer, summing operational amplifier

Blocks 39, 1602, 1702, 2002, 2102: Excitation function blocks

210,310,410,510,710,3711,3811,3911,4011,4111,4211,4311: Memory cell units

900, 1000, 1100, 1200, 1300, 2200, 2300, 2400, 2500, 2600, 2700, 2800, 2900, 3000: Multiplication array of neural vector matrices

901, 1003, 1103, 1203, 1303: Memory arrays

902, 1001, 1002, 1101, 1102, 1201, 1202, 1301, 1302: Reference Array

903,CG0,CG1,CG2,CG3,CG ₀ ,CG ₁ ,CG _M-1 ,CG _M ,CG _N : Control gate electrode wires

904,EG0,EG1,EGR0,EGR1: Remove gate polarity wires

1014, 1212, 1314: Diode-connected through-type multiplexer

1204: Series Transistor

1205, 1709, 1710, 2104: Multiplexers

1400: Long Short-Term Memory

1401, 1402, 1403, 1404, 1801, 1802, 1803, 1804: Cells

1500, 1600, 1700: Long Short-Term Memory Cells

1501, 1502, 1503, 1901, 1902: S-shaped function device

1504, 1505, 1903: Hyperbolic Tangent Device

1506, 1507, 1508, 1703, 1904, 1905, 1906, 2103: Multiplier device

1509, 1708, 1907, 2105: Adding device

1704, 1705, 1706, 1707, 2106, 2107, 2108: Registers

1800: Gate Control Return Unit

1900, 2000, 2100: Gate return cell

1908, 2109: Complementary Device

2701-1, 2701-2, 2701-(N-1), 2701-N: Bit line control gate

3100, 3210, 3300, 3400: Vector-matrix multiplication system

3101, 3102, 3213, 3303, 3304, 3305, 3306, 3307, 3308: Summation circuit

3211: First Line

3212: Second Formation

3401: Vector-Matrix Multiplication Array

3402: Column decoder

3403: High Voltage Decoder

3404: Line decoder

3405: Bitline Driver

3406: Input Circuit

3407, 3500, 4800, 4900, 5000: Output circuits

3408: Control Logic

3409: Bias Generator

3410: High Voltage Generation Block

3411: Electric Charge Pump

3412: Electric Charge Pump Regulator

3413: High Voltage Position Generator

3414: Algorithm Controller

3415: Analog Circuit System

3416: Control Engine

3417: Test Control Logic

3418: Static Random Access to Memory Blocks

3501, 3501-1, 3501-2, 3501-(j-1), 3501-j: Line Multiplexer

3502, 3502-1, 3502-2, 3502-(j-1), 3502-j: Analog-to-digital converters

3600, 3700, 3800, 3900, 4000, 4100, 4200, 4300, 4802, 4902, 5001, 5002: Σ-△ Analog-to-Digital Converters

3701,3801,3901,4001,4101,4201,4301: comparator

3702, 3802, 3902, 4002, 4102, 4202, 4302: State Machines

3703: Current Source

3704, 3804, 3805, 3904, 4005, 4104, 4105, 4204, 4205, 4304, 4305: Switches

3710, CFG_CREF, CFG_VREFSUP, CLK_REF, CLK_REFB, COMP_OUT: Signal

3803, 4003, 4103, 4203: Capacitors

3903: Adjustable Current Source

4006: Variable Reference Power Supply

4106: Transistor

4107: Gate Driver

4303: Variable Capacitor

4306: Variable Virtual BL Capacitor

4400, 4410: Waveforms

4500, 4600, 4700: Curve graph

4601, 4602, 4603: Voltage Range

4604, 4605, 4606, 4607, 4608, 4609: Current range

4801, 4901: Current-to-voltage converters

5003: Up/Down Counter

BL0, BL1, BL2, BL3, BLN, BL ₀ , BL _N-1 , BL _N , BL _i : bit lines

BLR0, BLR1, BLR2, BLR3: terminals

BLx,CGx,Egx,SLx,WLx,Inputx,INPUT ₀ ,INPUT ₁ ,INPUT ₂ ,INPUT ₃ ,INPUT ₄ ,INPUT _M-1 ,INPUT _M ,INPUT _N-1 ,INPUT _N : input

_c0 , _c1 , _c2 , c(t-1), c(t): Cell state vector

C1, C2, C3, S1, S2: Layers

CB1, CB2, CB3, CB4: synapses

CG0, CG1, CG2, CG3: Voltage inputs, controlling the gate voltage.

CLK: Clock signal

DOUT[7:0], DOUT[n:1], DOUTx[n:1]: Digital output Dual output D ...ual output Dual output Dual output Dual output Dual output Dual output Dual output Dual output Dual output D

EN: Enable signal

_h0 , _h1 , _h2 , _h3 , h(t-1), h(t): Output vector

IBL: Array Current, Input Current

IBL+, IBL-: Bit line current

IREF: Reference Current

IREFV: Variable Reference Current

ITV_Ox,VBL,VDD,VREFSUP: Voltage

ITV_O+, ITV_O-: Differential voltage

OUTPUT ₀ , OUTPUT ₁ , OUTPUT ₂ , OUTPUT ₃ , OUTPUT ₄ , OUTPUT _N-1 , OUTPUT _N : Output

P1, P2: Incentive functions

S0: Input layer

S3: Output Layer

SL0,SL1,SL ₀ ,SL ₁ ,SL ₂ ,SL ₃ ,SL _i ,SL _N : source line

VREF: Reference Voltage

WL,WL0,WL1,WL2,WL3,WL ₀ ,WL ₁ ,WL ₂ ,WL ₃ ,WL ₄ ,WL ₅ ,WL ₆ ,WL ₇ ,WL _M-1 ,WL _M ,WLA0,WLB0,WLA1,WLB1,WLA2,WLB2,WLA3,WLB3: word line

_x0 , _x1 , _x2 , _x3 , x(t): Input vector

Figure 1 is a diagram illustrating an artificial neural network.

Figure 2 depicts the separation of gated ultra-fast flash memory cells using prior art techniques.

Figure 3 depicts another prior art technique for isolating gate-type ultra-fast flash memory cells.

Figure 4 depicts another prior art technique for isolating gate-type ultra-fast flash memory cells.

Figure 5 depicts another prior art technique for isolating gate-type ultra-fast flash memory cells.

Figure 6 illustrates different levels of an artificial neural network utilizing one or more non-volatile memory arrays.

Figure 7 is a block diagram illustrating the VMM system.

Figure 8 is a block diagram illustrating an example of an artificial neural network utilizing one or more VMM systems.

Figure 9 illustrates another example of a VMM system.

Figure 10 depicts another example of a VMM system.

Figure 11 illustrates another example of a VMM system.

Figure 12 illustrates another example of a VMM system.

Figure 13 illustrates another example of a VMM system.

Figure 14 illustrates a prior art Long Short-Term Memory (LSTM) system.

Figure 15 depicts an instance cell used in a Long Short-Term Memory (LSTM) system.

Figure 16 illustrates an example implementation scheme of the cell in Figure 15.

Figure 17 illustrates another implementation scheme of the cell in Figure 15.

Figure 18 illustrates a prior art gate return unit system.

Figure 19 depicts an instance cell used in a gate control return unit system.

Figure 20 illustrates an example implementation of the cell from Figure 19.

Figure 21 illustrates another implementation scheme of the cell in Figure 19.

Figure 22 illustrates another example of a VMM system.

Figure 23 illustrates another example of a VMM system.

Figure 24 illustrates another example of a VMM system.

Figure 25 depicts another example of a VMM system.

Figure 26 illustrates another example of a VMM system.

Figure 27 illustrates another example of a VMM system.

Figure 28 illustrates another example of a VMM system.

Figure 29 illustrates another example of a VMM system.

Figure 30 depicts another example of a VMM system.

Figure 31 depicts another example of a VMM system.

Figure 32 depicts another example of a VMM system.

Figure 33 illustrates another example of a VMM system.

Figure 34 depicts another example of a VMM system.

Figure 35 illustrates the output circuit.

Figure 36 illustrates a Σ-Δ analog-to-digital converter.

Figure 37 depicts another Σ-Δ analog-to-digital converter.

Figure 38 depicts another Σ-Δ analog-to-digital converter.

Figure 39 depicts another Σ-Δ analog-to-digital converter.

Figure 40 depicts another Σ-Δ analog-to-digital converter.

Figure 41 depicts another Σ-Δ analog-to-digital converter.

Figure 42 depicts another Σ-Δ analog-to-digital converter.

Figure 43 depicts another Σ-Δ analog-to-digital converter.

Figures 44A and 44B depict the waveforms of a Σ-Δ analog-to-digital converter.

Figure 45 depicts the curves of bit line voltage and bit line current.

Figure 46 depicts the bit line voltage and bit line current curves for various values of Qref.

Figure 47 depicts the curves of bit line voltage and bit line current.

Figure 48 illustrates the output circuit.

Figure 49 illustrates the output circuit.

Figure 50 illustrates the output circuit.

VMM System Architecture

Figure 34 depicts a block diagram of a VMM system 3400. The VMM system 3400 includes a VMM array 3401, a column decoder 3402, a high-voltage decoder 3403, a row decoder 3404, a bit line driver 3405 (such as for a programmable bit line control circuit system), an input circuit 3406, an output circuit 3407, a control logic 3408, and a bias generator 3409. The VMM system 3400 further includes a high-voltage generation block 3410, which includes a charge pump 3411, a charge pump regulator 3412, and a high-voltage level generator 3413. The VMM system 3400 further includes a (programmed/erase or weighted tuning) algorithm controller 3414, an analog circuit system 3415, a control engine 3416 (which may include, but is not limited to, functions such as arithmetic functions, excitation functions, and embedded microcontroller logic), a test control logic 3417, and a static random access memory (SRAM) block 3418 for storing data such as intermediate data for input circuits (e.g., excitation data) or intermediate data for output circuits (neural output data, partial and output neuron data), or for programmed data input (e.g., for a whole column or for multiple columns of data input).

Input circuitry 3406 may include circuitry such as a DAC (digital-to-analog converter), DPC (digital-to-pulse converter, digital-to-time-modulation pulse converter), AAC (analog-to-analog converter, such as a current-to-voltage converter, logarithmic converter), PAC (pulse-to-analog level converter), or any other type of converter. Input circuitry 3406 may implement one or more of a normalization, linear or nonlinear scaling, or arithmetic functions. Input circuitry 3406 may implement a temperature compensation function for the input level. Input circuitry 3406 may implement an excitation function, such as ReLU or S-type. Input circuit 3406 can store digital excitation data that will be applied as an input signal or combined with an input signal during programming or read operations. The digital excitation data can be stored in a register. Input circuit 3406 may include circuitry for driving array terminals such as CG, WL, EG, and SL lines, and may include sample-and-hold circuitry and a buffer. A DAC can be used to convert the digital excitation data into an analog input voltage to be applied to the array.

Output circuit 3407 may include circuits such as ITV (current-to-voltage circuit), ADC (analog-to-digital converter, used to convert analog neuron outputs to digital bits), AAC (analog-to-analog converter, such as current-to-voltage converter, logarithmic converter), APC (analog-to-pulse converter, analog-to-time modulation pulse converter), or any other type of converter. Output circuit 3407 can convert array outputs into excitation data. Output circuit 3407 can implement excitation functions such as rectified linear excitation function (ReLU) or sigmoid function. Output circuit 3407 can implement one or more of the following for neural outputs: statistical normalization, regularization, scaling/scaling/gain functions, statistical rounding, or arithmetic functions (e.g., addition, subtraction, division, multiplication, shifting, logarithmic). Output circuit 3407 can implement temperature compensation functions for neural outputs or array outputs (such as bit-line outputs) to keep the array's power consumption approximately constant over a temperature range or to improve the accuracy of the array (neuron) output, for example, by keeping the IV slope approximately the same over a temperature range. Output circuit 3407 may include registers for storing output data.

Figure 35 illustrates output circuit 3500. Output circuit 3500 is an example implementation of output circuit 3407 in Figure 34. Output circuit 3500 includes row multiplexers 3501-1, 3501-2, ..., 3501-(j-1), 3501-j and analog-to-digital converters (ADCs) 3502-1, 3502-2, ..., 3502-(j-1), 3502-j. Each row multiplexer 3501 receives current, such as bit line current, from one or more rows of the VMM array 3401. If more than one row is connected to a separate row multiplexer 3501, the row multiplexer 3501 selects a row and supplies the current from that row to a separate ADC 3502, which converts the received current into a digital output DOUTx[n:1], where x is the number of rows and DOUT contains n bits. If j equals the number of rows in the VMM array 3401, meaning that each row has its own ADC 3502, then the row multiplexer 3501 is selectable, and each row in the VMM array 3401 can be directly connected to its associated ADC 3502.

Figure 36 illustrates the Σ-Δ ADC 3600, which can be used in the ADC 3502 of the output circuit 3500 in Figure 35. The Σ-Δ ADC 3600 receives the current IBL, enable signal EN, and clock signal CLK from the row of VMM array 3401, which is illustrated for simplicity as a single memory cell. CLK will typically have a higher clock frequency than other circuit systems used in VMM systems because the Σ-Δ ADC 3600 uses CLK to sample the current received from VMM array 3401. The Σ-Δ ADC 3600 output consists of an n-bit digital output DOUT[n:1]. The Σ-△ ADC 3600 directly converts the array current IBL into digital output bits without using a current-to-voltage converter (ITV).

Figure 37 illustrates a Σ-Δ ADC 3700, which is an example of the Σ-Δ ADC 3600 in Figure 36. The Σ-Δ ADC 3700 includes a comparator 3701, a state machine (SM) 3702, a current source 3703, and a switch 3704. The Σ-Δ ADC 3700 is coupled to a row in a VMM array 3401, which for simplicity is illustrated as a single memory cell 3711 that draws current from the Σ-Δ ADC 3700. The SM 3702 can be implemented using discrete logic, a programmable device, a processor, a controller, or other mechanisms. The Σ-Δ ADC 3700 is coupled to a row in the VMM array 3401, which for simplicity is described as a single memory cell that draws current from the Σ-Δ ADC 3700. The Σ-Δ ADC 3700 also receives an enable signal EN, a clock signal CLK, and a reference voltage VREF. Current source 3703 provides the reference current IREF and is an example of an injection circuit. Switch 3704 is controlled by SM 3702. Comparator 3701 includes a first terminal (inverting terminal) coupled to the row (described as memory cell 3711) and coupled to the current source 3703 via switch 3704, and a second terminal (non-inverting terminal) coupled to the reference voltage VREF. SM 3702 receives the output of comparator 3701. When the voltage at the bit line (BL) node of memory cell 3711 is less than VREF, the clock signal CLK is transmitted by SM 3702 or converted by SM 3702 into a converted clock signal, CLK_REF 3710. This signal CLK_REF 3710 therefore has a clock pulse with the same or opposite polarity as the clock signal CLK. If the clock signal CLK has been converted by SM 3702, the clock pulse of the signal CLK_REF 3710 may have a different operating period or polarity than the clock pulse of the clock signal CLK. This signal CLK_REF 3710 controls switch 3704. When the signal CLK_REF is established during each clock cycle, switch 3704 closes, allowing Iref from current source 3703 to flow to the BL node of memory cell 3711, thereby increasing the voltage on the BL node or memory cell 3711. In response to the clock signal CLK, the signal CLK_REF continues to count down, and upon the establishment of the signal CLK_REF, a current IREF is injected into the BL node of memory cell 3711. That is, a charge Qref proportional to IREF * T is injected into the BL node or memory cell 3711, where T is the establishment time of the signal CLK_REF. This continues until the voltage on the BL node of memory cell 3711 is greater than VREF. At this time, SM 3702 will keep the signal CLK_REF de-established. This means that the CLK pulse will not be transmitted from SM 3702 to the signal CLK_REF or converted by SM 3702 for transmission to the signal CLK_REF. In response to the start of a read operation, VMM array 3401 draws current to bring the BL node of memory cell 3711 low. In the absence of additional charge, since the signal CLK_REF is de-established, VMM array 3401 draws current to bring the BL node of memory cell 3711 low until it < VREF, and then repeats the procedure. SM 3702 tracks the timing on the signal CLK_REF and generates the digital output bit DOUT[n:1] in response. In this example, DOUT[n:1] from SM 3702 is a digital representation of the count of the number of times switch 3704 is closed during a certain number of cycles of clock signal CLK, and reflects the current drawn by VMM array 3401. For example, an 8-bit digital output DOUT[7:0] can be output over 512 CLK cycles. A high value indicates that switch 3704 is opened and closed a relatively large number of times, meaning that the VMM array 3401 is drawing a large current because the charge supplied by current source 3703 to the BL node of memory cell 3711 to match the current drawn by the VMM array 3401 is therefore relatively large. A low value indicates that switch 3704 is opened and closed a relatively small number of times, meaning that the VMM array 3401 is drawing a small current. The digital output value reflects the total charge (i.e., the amount of Qref) injected into the BL to balance the array current (to keep the BL constant). A larger current produces a larger digital output value, and a smaller current produces a smaller digital output value. DOUT[n:1] indicates the value of the current drawn from the row (described as memory cell 3711).

Figure 38 illustrates a Σ-Δ ADC 3800, which is an example of the Σ-Δ ADC 3600 in Figure 36. The Σ-Δ ADC 3800 includes a comparator 3801, an SM 3802, a capacitor CREF 3803, and switches 3804 and 3805. The Σ-Δ ADC 3800 is coupled to a row in a VMM array 3401, which for simplicity is illustrated as a single memory cell 3811 that draws current from the Σ-Δ ADC 3800. SM 3802 provides control signals CLK_REFB and CLK_REF to switches 3804 and 3805, respectively. CLK_REFB and CLK_REF are logically opposite. Switch 3804 is used to charge capacitor 3803 to a voltage (VDD in this case) when it is closed (when CLK_REFB is confirmed), and switch 3805 discharges the charge from capacitor 3803 to the bit line (BL) node of individual memory cell 3811 when it is closed (when CLK_REF is confirmed), wherein the reference charge Qref on capacitor 3803 will be reduced proportionally to (VDD - voltage of BL node of cell 3811) * capacitance of capacitor 3803. That is, for each clock pulse in signals CLK_REF and CLK_REFB, a reference charge Qref is injected into the BL node of cell 3811, whereby the reference charge flows to ground along with the current through memory cell 3811. Capacitor 3803 is an example of the injection circuit. Σ-Δ ADC 3800 is coupled to a row in VMM array 3401, drawing current from Σ-Δ ADC 3800. Comparator 3801 includes a first terminal (inverting terminal) coupled to the row (described as memory cell 3811) and coupled to capacitor 3803 via switch 3805, and a second terminal (non-inverting terminal) coupled to reference voltage VREF. SM 3802 receives the output of comparator 3801. The Σ-Δ ADC 3800 also receives the enable signal EN, the clock signal CLK, and the reference voltage VREF. By injecting a reference charge instead of a reference current into the BL node or memory cell 3811, capacitor 3803 will behave equivalently to the current source 3703 in Figure 37. The Σ-Δ ADC 3800 also operates in a similar manner to the Σ-Δ ADC 3700 in Figure 37, the main difference being that it uses a reference charge instead of a reference current. The output bits DOUT[n:1] from SM 3802 are the digital count of the number of times switch 3805 is closed during a certain number of cycles of the clock signal CLK. A higher value indicates that switch 3805 is opened and closed a relatively large number of times, meaning that VMM array 3401 is drawing a large current, while a lower value indicates that switch 3805 is opened and closed a relatively small number of times, meaning that VMM array 3401 is drawing a small current.

Figure 39 depicts the Σ-Δ ADC 3900, an example of the Σ-Δ ADC 3600 in Figure 36. The Σ-Δ ADC 3900 is similar to the Σ-Δ ADC 3700 in Figure 37, except that it uses an adjustable current source 3903 (an example of an injection circuit) that generates a variable reference current IREFV instead of a fixed reference current. Furthermore, the adjustable current source 3903 is controlled by SM 3902 or a global control signal (not shown). The adjustable current source 3903 can be adjusted to provide different amounts of reference current to compensate for any variations in array current ranges, bit line capacitances, or PVT (process temperature and voltage). Comparator 3901 includes a first terminal (inverting terminal) coupled to a row (described as memory cell 3911) and coupled to an adjustable current source 3903 via a switch 3904, and a second terminal (non-inverting terminal) coupled to a reference voltage VREF. SM 3902 receives the output of comparator 3901.

Figure 40 depicts the Σ-Δ ADC 4000, an example of the Σ-Δ ADC 3600 in Figure 36. The Σ-Δ ADC 4000 is similar to the Σ-Δ ADC 3800 in Figure 38, except that the voltage VREFSUP is supplied by a variable reference power supply 4006 to a reference capacitor 4003. Capacitor 4003 is an example of an injection circuit. Furthermore, the variable reference power supply 4006 responds to SM 4002 or generates a variable voltage via a global control signal (not shown). That is, the voltage supplied by the variable reference power supply 4006 can be adjusted in response to the signal CFG_VREFSUP, which can be provided by SM 4002. The voltage supplied by the variable reference power supply 4006 can be adjusted to compensate for any variations in array current ranges, bit-line capacitances, or PVT (process temperature and voltage). Comparator 4001 includes a first terminal (inverting terminal) coupled to a row (described as memory cell 4011) and via switch 4005 coupled to capacitor 4003, and a second terminal (non-inverting terminal) coupled to the reference voltage VREF. SM 4002 receives the output of comparator 4001.

Figure 41 illustrates a Σ-Δ ADC 4100, which is an example of the Σ-Δ ADC 3600 in Figure 36. The Σ-Δ ADC 4100 includes a comparator 4101, an SM 4102, a capacitor CREF 4103, switches 4104 and 4105, a transistor 4106, and a gate driver 4107 that drives the gate of the transistor 4106. The Σ-Δ ADC 4100 is similar to the Σ-Δ ADC 3800 in Figure 38, except for the use of transistor 4106, in which the gate of the transistor is controlled by a reference voltage provided by the gate driver 4107. Comparator 4101 includes a first terminal (inverting terminal) coupled to the row (described as memory cell 4111) and coupled to capacitor 4103 via switch 4105, and a second terminal (non-inverting terminal) coupled to a reference voltage VREF. SM 4102 receives the output of comparator 4101. Transistor 4106 controls how much charge is transferred (injected) into the BL node of memory cell 4111. The injected charge is proportional to (Vdd - (VREFX + Vt_4106)) * capacitance of capacitor 4103, where Vt_4106 is the threshold voltage of transistor 4106. VREFx can be controlled by SM 4102 or a global control signal (not shown). It can be used to compensate for any variations in array current range, bit line capacitance, or PVT (process temperature and voltage).

Figure 42 illustrates a Σ-Δ ADC 4200, which is an example of the Σ-Δ ADC 3600 in Figure 36. The Σ-Δ ADC 4200 includes a comparator 4201, an SM 4202, an adjustable capacitor 4203, switches 4204 and 4205. The Σ-Δ ADC 4200 is similar to the Σ-Δ ADC 4000 in Figure 40, except that it uses an adjustable capacitor 4203 instead of a variable reference power supply. The adjustable capacitor 4203 responds to the SM 3902 or generates a variable charge via a global control signal (not shown); that is, the capacitance provided by the adjustable capacitor 4203 responds to the signal CFG_CREF, which can be provided by the SM 4202. The signal CFG_CREF and the adjustable capacitor 4203 can be used to compensate for any variations in array current range, bit line capacitance, or PVT (process temperature and voltage). Comparator 4201 includes a first terminal (inverting terminal) coupled to the row (described as memory cell 4211) and via switch 4205 to the adjustable capacitor 4203, and a second terminal (non-inverting terminal) coupled to the reference voltage VREF. SM 4202 receives the output of comparator 4201.

Figure 43 depicts the Σ-Δ ADC 4300, which is an example of the Σ-Δ ADC 3600 in Figure 36. The Σ-Δ ADC 4300 includes a comparator 4301, an SM 4302, a variable capacitor 4303, a variable virtual BL capacitor 4306, a switch 4304, and a switch 4305. The Σ-Δ ADC 4300 is similar to the Σ-Δ ADC 4200, except that the variable virtual BL capacitor 4306 is added in parallel with the VMM array 3401. The variable virtual BL capacitor 4306 responds to SM 4302 or generates a variable charge via a global control signal (not shown). That is, the capacitance provided by the variable virtual BL capacitor 4306 can be varied in response to SM 4302 or via a global control signal (not shown). This can be used to compensate for any changes in different array current ranges, different bit line capacitances, or PVT (process temperature and voltage). Comparator 4301 includes a first terminal (here, the inverting terminal) coupled to the row (described as memory cell 4311) and coupled via switch 4305 to adjustable capacitors 4306 and 4303 (which together form an example of an injection circuit), and a second terminal (here, the non-inverting terminal) coupled to a reference voltage VREF. The SM 4302 receives the output of the comparator 4301.

Figures 44A and 44B illustrate example waveforms 4400 and 4410 of the operation of Σ-Δ ADCs 3600, 3700, 3800, 3900, 4000, 4100, 4200, and 4300. In waveform 4400, the VMM array 3401 draws current I1, while in waveform 4410, the VMM array 3401 draws half of that current I1/2. VBL is the voltage of the bit lines in the VMM array 3401, coupled to the inverting inputs of comparators 3701, 3801, 3901, 4001, 4101, 4201, and 4301, and VREF is the voltage provided to the non-inverting inputs of comparators 3701, 3801, 3901, 4001, 4101, 4201, and 4301. The clock signals CLK and CLK_REF are shown, with the outputs of comparators 3701, 3801, 3901, 4001, 4101, 4201, and 4301 described as the signal COMP_OUT. When individual switches 3704, 3805, 3904, 4005, 4105, 4205, and 4305 are closed, VBL increases, and subsequently decreases when those switches are opened. The number of times each switch 3704, 3805, 3904, 4005, 4105, 4205, and 4305 is closed (or opened) is counted, and the count value for a given period is output as the digital output DOUT[n:1]. In waveform 4400, the count is 4 during the 16 time periods of the clock signal CLK shown. In waveform 4410, the count is 2 during the 16 time periods of the clock signal CLK shown, because the array current is half of the current in waveform 4400 in Figure 44A. The relevant time intervals for measurement and counting can be greater than or less than the 16 time intervals of the clock signal CLK.

Figure 45 illustrates the curves 4500 of the operation of the Σ-Δ ADCs 3600, 3700, 3800, 3900, 4000, 4100, 4200, and 4300, respectively. Curve 4500 shows the relationship between VBL and IBL, where VBL is the voltage at the BL node of memory cells 3711, 3811, 3911, 4011, and 4111, respectively, and IBL is the current drawn from the BL node by the VMM array 3401 through the inputs of comparators 3701, 3801, 3901, 4001, 4101, 4201, and 4301, respectively. As shown, large VBL variations occur within the array current range, therefore improvements are needed to reduce these voltage variations within the demonstrated current range.

Figure 46 illustrates the curves 4600 for the operation of the Σ-Δ ADCs 3600, 3700, 3800, 3900, 4000, 4100, 4200, and 4300. Curve 4600 shows the relationship between VBL and IBL, where VBL is the voltage at the BL nodes of memory cells 3711, 3811, 3911, 4011, and 4111, respectively, and IBL is the current drawn from the BL nodes by the VMM array 3401 through the inputs of comparators 3701, 3801, 3901, 4001, 4101, 4201, and 4301.

Curve 4600 depicts the traces of VBL and IBL for different values of Qref, where Qref is the reference charge injected into the bit line. Here, traces of 21 Qref examples are shown: QREF1, QREF2, ..., QREF20, QREF21. These are examples, and any number of different Qref values can be used. SMs 3702, 3802, 3902, 4002, 4102, 4202, and 4302 can be programmed to modify Qref based on a range of the received or expected input current IBL. The following algorithm can be used to select the Qref value for each current range.

First, the possible voltage range of VBL is divided into multiple ranges. Here, three example voltage ranges are shown: voltage ranges 4601, 4602, and 4603. Within each voltage range, the acceptable error is determined. For example, for voltage range 4602 (VBL), the voltage varies between approximately 599mV and 601.5mV, and the acceptable error is 2.5mV.

Second, considering this acceptable error, the possible current range of the IBL is divided into multiple ranges. Here, six example current ranges are shown: current ranges 4604, 4605, 4606, 4607, 4608, and 4609. Within each current range, one or more Qref values are identified, which generate voltage traces within the acceptable error range of the voltage range; alternatively, unacceptable Qref values are identified.

For example, within current range 4604, QREF1 through QREF4 may be acceptable because their traces are fairly linear, but some other QREFs may be unacceptable because they exhibit nonlinearity near the beginning or end of the range or outside the acceptable range 4602.

Within current range 4605, QREF1, QREF2, and QREF3 may be considered unacceptable because their traces exhibit nonlinearity (e.g., large drops) within that current range. QREF4 through QREF9 are acceptable, while all other QREFs may be considered unacceptable because their traces exhibit nonlinearity or are outside voltage range 4602.

Within the current range 4606, QREF1 through QREF8 may be considered unacceptable because they do not even operate within the voltage range 4602, while QREF10 through QREF13 are acceptable within the range 4602 and exhibit fairly linear traces.

Similarly, within the current range 4607, QREF1 to QREF12 may be considered unacceptable, while QREF13 to QREF16 are acceptable.

Similarly, within the current range 4608, QREF1 through QREF16 may be considered unacceptable, while QREF17 through QREF20 are acceptable.

Similarly, within the current range 4609, QREF1 to QREF19 may be considered unacceptable, while QREF20 to QREF21 are acceptable.

Because smaller QREFs are easier and faster to generate (e.g., due to capacitor charging time), SMs 3702, 3802, 3902, 4002, 4102, 4202, and 4302 can select the smallest acceptable QREF for each range. For example, SMs 3702, 3802, 3902, 4002, 4102, 4202, and 4302 can select QREF3 for current range 4604, QREF8 for current range 4605, QREF12 for current range 4606, QREF16 for current range 4607, QREF20 for current range 4608, and QREF21 for current range 4609.

Figure 47 illustrates the curves 4700 of the operation of the Σ-Δ ADCs 3600, 3700, 3800, 3900, 4000, 4100, 4200, and 4300. Curve 4700 shows the relationship between VBL and IBL, where VBL is the voltage at the BL node of memory cells 3711, 3811, 3911, 4011, and 4111, respectively, and IBL is the current drawn from the BL node by the VMM array 3401 through the inputs of comparators 3701, 3801, 3901, 4001, 4101, 4201, and 4301, respectively. As shown, the current range is divided into six zones, and different QREFs are used in each zone by adjusting the values of CREF, IREF, and VSUPREF based on the Σ-Δ ADC design. SMs 3702, 3802, 3902, 4002, 4102, 4202, and 4302 can select specific CREF/IREF/VSUPREF values based on the measured current range of the VMM array, for example, by monitoring digital output bits to determine the current range based on the values of those bits, in order to achieve a more linear range among the various options illustrated in the traces of curves 4500 and 4600, or options not shown. Alternatively, these values can be changed by the SM during ADC operation.

Figure 48 illustrates the output circuit 4800, which includes a current-to-voltage converter 4801 and a Σ-Δ ADC 4802. The current-to-voltage converter 4801 receives bit-line current from a VMM array 3401 (not shown) and converts the current into a voltage ITV_Ox, which is then provided to the SD ADC 4802, which converts the voltage into a digital output [n:1].

Figure 49 illustrates the output circuit 4900, which includes a current-to-voltage converter 4901 and a Σ-Δ ADC 4902. The current-to-voltage converter 4901 receives different bit-line currents IBL+ and IBL- from a VMM array 3401 (not shown) and converts these differential currents into differential voltages ITV_O+ and ITV_O-. These differential voltages ITV_O+ and ITV_O- are then provided to the SD ADC 4902, which converts these voltages into a digital output [n:1]. In this case, the Σ-Δ ADC (not shown) operates on voltage, not current.

Figure 50 depicts a differential output circuit 5000, where the digital output bits represent differential weights, for example, W = (W+) - (W-) or IBL = (IBL+) - (IBL-). Σ-Δ ADCs 5001 and 5002 can be any of the Σ-Δ ADCs 3600, 3700, 3800, 3900, 4000, 4100, 4200, and 4300. During operation, IBL+ is converted to digital bits by the Σ-Δ ADC 5001, and the result is stored in the up/down counter 5003 by counting up from the midpoint value. Then, IBL- is converted to digital bits by the Σ-Δ ADC 5002, and the result is used to count down the value in counter 5003. Therefore, the final value DOUT[n:1] output by the up/down counter 5003 is the differential weight represented by (IBL+)-(IBL-). For example, if the up/down counter 5003 is an 8-bit counter, the intermediate value could be 127 (01000000). Up counting using the output of the Σ-Δ ADC 5001 will produce a value between 127 and 255, and down counting will produce a final value between 0 and 255, where 128 to 255 represent positive weights and 0 to 127 represent negative weights.

As used herein, the terms "above" and "on" inclusively include "directly on" (without intermediate materials, components, or spaces) and "indirectly on" (with intermediate materials, components, or spaces). Similarly, the term "adjacent" includes "directly adjacent" (without intermediate materials, components, or spaces) and "indirectly adjacent" (with intermediate materials, components, or spaces), "installed to" includes "directly installed to" (without intermediate materials, components, or spaces) and "indirectly installed to" (with intermediate materials, components, or spaces), and "electrically coupled" includes "directly electrically coupled to" (without intermediate materials or components electrically connecting the components together) and "indirectly electrically coupled to" (with intermediate materials or components electrically connecting the components together). For example, forming an element "above a substrate" can include forming the element directly on the substrate without intermediate materials/elements, and forming the element indirectly on the substrate with one or more intermediate materials/elements.

Claims (26)

A system for generating a digital output from a vector matrix multiplication array includes: a vector matrix multiplication array comprising an array of nonvolatile memory cells arranged in columns and rows; and a Σ-Δ analog-to-digital converter for receiving a current and a clock signal from a row of the vector matrix multiplication array and generating a digital output based on a sampling of the current in response to the clock signal. The system of claim 1, wherein the Σ-Δ analog-to-digital converter comprises: a current source; a switch; a comparator including a first input terminal coupled to the row and coupled to the current source via the switch, a second input terminal coupled to a reference voltage, and an output terminal for providing an output; and a state machine for receiving the output from the comparator, providing a control signal to the switch, and responding to the current to generate the digital output, the digital output indicating a value of the current drawn from the row. The system of claim 1, wherein the Σ-Δ analog-to-digital converter comprises: a capacitor; a switch; a comparator including a first input terminal coupled to the row and coupled to the capacitor via the switch, a second input terminal coupled to a reference voltage, and an output terminal for providing an output; and a state machine for receiving the output from the comparator to provide a control signal to the switch, and responding to the current to generate the digital output, the digital output indicating a value of the current drawn from the row. The system of claim 1, wherein the Σ-Δ analog-to-digital converter comprises: an adjustable current source; a switch; a comparator including a first input terminal coupled to the row and coupled to the adjustable current source via the switch, a second input terminal coupled to a reference voltage, and an output terminal for providing an output; and a state machine for receiving the output from the comparator to provide a control signal to the switch, and responding to the current to generate the digital output, the digital output indicating a value of the current drawn from the row. The system of claim 1, wherein the Σ-Δ analog-to-digital converter comprises: a variable voltage source; a capacitor; a switch; a comparator including a first input terminal coupled to the row and coupled to the capacitor via the switch, a second input terminal coupled to a reference voltage, and an output terminal for providing an output; and a state machine for receiving the output from the comparator to provide a first control signal to the switch and a second control signal to the variable voltage source, and for responding to the current to generate the digital output, the digital output indicating a value of the current drawn from the row. The system of claim 1, wherein the Σ-Δ analog-to-digital converter comprises: a capacitor; a switch; a transistor; a control circuit; a comparator including a first input terminal coupled to the row and coupled to the capacitor via the switch and the transistor, a second input terminal coupled to a reference voltage, and an output terminal for providing an output; and a state machine for receiving the output from the comparator to provide a first control signal to the switch and a second control signal to the control circuit, and for responding to the current to generate the digital output, the digital output indicating a value of the current drawn from the row. The system of claim 1, wherein the Σ-Δ analog-to-digital converter comprises: an adjustable capacitor; a switch; a comparator including a first input terminal coupled to the row and coupled to the adjustable capacitor via the switch, a second input terminal coupled to a reference voltage, and an output terminal for providing an output; and a state machine for receiving the output from the comparator to provide a first control signal to the switch and a second control signal to the adjustable capacitor, and for responding to the current to generate the digital output, the digital output indicating a value of the current drawn from the row. The system of claim 7, wherein the Σ-Δ analog-to-digital converter includes: a second adjustable capacitor coupled to the first input terminal of the comparator. A system for generating a digital output from a vector matrix multiplication array includes: a vector matrix multiplication array comprising an array of nonvolatile memory cells configured in columns and rows; and an analog-to-digital converter for receiving a current and a clock signal from a row of the vector matrix multiplication array and generating a digital output based on a sampling of the current in response to the clock signal, wherein a variable charge or a variable current is injected by an injection circuit to maintain a voltage of the row approximately constant during operation. The system of claim 9, wherein the variable charge or the variable current is selected based on the range of current measured by one of the arrays. The system of claim 10, wherein the variable charge or the variable current varies during the operation of the analog-to-digital converter. The system of claim 9, wherein the variable charge is provided by a variable voltage source coupled to a capacitor. The system of claim 9, wherein the variable charge is changed by changing the capacitance value of a variable capacitor. The system of claim 9 includes an up/down counter to determine a difference weight provided by two rows in the vector matrix multiplication array. The system, as in claim 14, includes a second analog-to-digital converter. The system of claim 15, wherein the analog-to-digital converter generates the up/down counter for an up count of a first value, and the second analog-to-digital converter generates the up/down counter for a down count of a second value. The system of claim 9, wherein the analog-to-digital converter is a Σ-Δ analog-to-digital converter. A method for generating a digital output from a vector matrix multiplication array, comprising: using a clock signal to sample the current from a nonvolatile memory cell array configured in columns and rows to convert the current into digital output bits, the conversion comprising injecting a variable charge or a variable current until the voltage of one row is equal to a reference voltage within a target voltage range. The method of request 18, wherein the array is used for vector-matrix multiplication. The method of claim 18, wherein the variable charge is provided by an adjustable capacitor. The method of claim 18, wherein the variable charge is provided by a capacitor, wherein a variable supply reference voltage is supplied at one terminal of the capacitor. The method of claim 18, wherein the variable charge is provided by a plurality of capacitors. The method of claim 18, wherein the conversion is performed by a Σ-Δ analog-to-digital converter. The method of claim 18 includes: selecting a value of the variable charge or a value of the variable current based on a measured or expected current range of the current. The method of claim 18 includes: selecting a value of the variable charge based on a expected voltage range of the nonvolatile memory cell. The method of claim 18 includes: selecting a value of the variable charge based on a expected current range of the nonvolatile memory cell.