[go: up one dir, main page]

WO2022005673A1 - Opérations neuromorphiques utilisant des positons - Google Patents

Opérations neuromorphiques utilisant des positons Download PDF

Info

Publication number
WO2022005673A1
WO2022005673A1 PCT/US2021/035125 US2021035125W WO2022005673A1 WO 2022005673 A1 WO2022005673 A1 WO 2022005673A1 US 2021035125 W US2021035125 W US 2021035125W WO 2022005673 A1 WO2022005673 A1 WO 2022005673A1
Authority
WO
WIPO (PCT)
Prior art keywords
format
data
memory
analog
operations
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/US2021/035125
Other languages
English (en)
Inventor
Vijay S. Ramesh
Richard C. Murphy
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Micron Technology Inc
Original Assignee
Micron Technology Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US16/915,039 external-priority patent/US11636323B2/en
Priority claimed from US16/997,599 external-priority patent/US12159219B2/en
Application filed by Micron Technology Inc filed Critical Micron Technology Inc
Priority to EP21833816.8A priority Critical patent/EP4172875A4/fr
Priority to KR1020237002393A priority patent/KR102813842B1/ko
Priority to CN202180035602.9A priority patent/CN115668224B/zh
Publication of WO2022005673A1 publication Critical patent/WO2022005673A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F7/00—Methods or arrangements for processing data by operating upon the order or content of the data handled
    • G06F7/38—Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation
    • G06F7/48—Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation using non-contact-making devices, e.g. tube, solid state device; using unspecified devices
    • G06F7/544—Methods or arrangements for performing computations using exclusively denominational number representation, e.g. using binary, ternary, decimal representation using non-contact-making devices, e.g. tube, solid state device; using unspecified devices for evaluating functions by calculation
    • G06F7/5443—Sum of products
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11C—STATIC STORES
    • G11C11/00—Digital stores characterised by the use of particular electric or magnetic storage elements; Storage elements therefor
    • G11C11/54—Digital stores characterised by the use of particular electric or magnetic storage elements; Storage elements therefor using elements simulating biological cells, e.g. neuron
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs

Definitions

  • the present disclosure relates generally to semiconductor memory and methods, and more particularly, to apparatuses, systems, and methods for neuromorphic operations using posits.
  • Memory devices are typically provided as internal, semiconductor, integrated circuits in computers or other electronic systems. There are many different types of memory including volatile and non-volatile memory. Volatile memory can require power to maintain its data (e.g., host data, error data, etc.) and includes random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), synchronous dynamic random access memory (SDRAM), and thyristor random access memory (TRAM), among others.
  • RAM random access memory
  • DRAM dynamic random access memory
  • SRAM static random access memory
  • SDRAM synchronous dynamic random access memory
  • TAM thyristor random access memory
  • Non-volatile memory can provide persistent data by retaining stored data when not powered and can include NAND flash memory, NOR flash memory, and resistance variable memory such as phase change random access memory (PCRAM), resistive random access memory (RRAM), and magnetoresistive random access memory (MRAM), such as spin torque transfer random access memory (STT RAM), among others.
  • PCRAM phase change random access memory
  • RRAM resistive random access memory
  • MRAM magnetoresistive random access memory
  • STT RAM spin torque transfer random access memory
  • Memory devices may be coupled to a host (e.g., a host computing device) to store data, commands, and/or instructions for use by the host while the computer or electronic system is operating. For example, data, commands, and/or instructions can be transferred between the host and the memory device(s) during operation of a computing or other electronic system.
  • a host e.g., a host computing device
  • data, commands, and/or instructions can be transferred between the host and the memory device(s) during operation of a computing or other electronic system.
  • Figure 1 is a functional block diagram in the form of a computing system including an apparatus including a host and a memory device in accordance with a number of embodiments of the present disclosure.
  • Figure 2A is another functional block diagram in the form of a computing system including an apparatus including a host and a memory device in accordance with a number of embodiments of the present disclosure.
  • Figure 2B is a functional block diagram in the form of a computing system including a host, a memory device, an application-specific integrated circuit, and a field programmable gate array in accordance with a number of embodiments of the present disclosure.
  • Figure 3 is an example of an «-bit post with es exponent bits.
  • Figure 4A is an example of positive values for a 3-bit posit.
  • Figure 4B is an example of posit construction using two exponent bits.
  • Figure 5 is a functional block diagram in the form of a neuromorphic memory array, a digital to analog converter, and an analog to digital data converter in accordance with a number of embodiments of the present disclosure.
  • Figure 6 is a functional block diagram in the form of control circuitry in accordance with a number of embodiments of the present disclosure.
  • Figure 7 is a flow diagram representing an example method for neuromorphic operations using posits in accordance with a number of embodiments of the present disclosure.
  • An example system may include a memory device and the memory device may include a plurality of memory cells.
  • the plurality of memory cells can store data including a bit string in an analog format.
  • a neuromorphic operation can be performed on the data in the analog format.
  • the example system may include an analog to digital converter coupled to the memory device.
  • the analog to digital converter may convert the bit string in the analog format stored in at least one of the plurality of memory cells to a format that supports arithmetic operations to a particular level of precision.
  • the memory device may include control circuitry.
  • the control circuitry may include a memory resource and a processing resource.
  • the control circuitry may be coupled to a neuromorphic memory array.
  • the memory array can store data including a bit string in an analog format.
  • the control circuitry can control performance of neuromorphic operations on the bit string in the analog format.
  • the neuromorphic memory array can act as a neural network.
  • Analog weights can be input into operation of the neural network in order to train the neural network.
  • the results of the neuromorphic operations can be converted to data in a non-analog format (e.g., a format that supports arithmetic operations such as a posit format, as will be described further below).
  • the non-analog format can be a universal number (unum) format, such as a Type III unum or posit format.
  • a neural network can include a set of instructions that can be executed to recognize patterns in data. Some neural networks can be used to recognize underlying relationships in a set of data in a manner that mimics the way that a human brain operates. A neural network can adapt to varying or changing inputs such that the neural network can generate a best possible result in the absence of redesigning the output criteria.
  • a neural network can consist of multiple neurons, which can be represented by one or more equations.
  • a neuron can receive a quantity of numbers or vectors as inputs and, based on properties of the neural network, produce an output.
  • a neuron can receive Xk inputs, with k corresponding to an index of input.
  • the neuron can assign a weight vector, Wk, to the input.
  • the weight vectors can, in some embodiments, make the neurons in a neural network distinct from one or more different neurons in the network.
  • respective input vectors can be multiplied by respective weight vectors to yield a value, as shown by Equation 1, which shows and example of a linear combination of the input vectors and the weight vectors.
  • a non-linear function e.g., an activation function
  • an activation function can be applied to the value / (xi, xi) that results from Equation 1.
  • An example of a non-linear function that can be applied to the value that results from Equation 1 is a rectified linear unit function (ReLU).
  • ReLU rectified linear unit function
  • Equation 2 Application of the ReLU function, which is shown by Equation 2, yields the value input to the function if the value is greater than zero, or zero if the value input to the function is less than zero.
  • the ReLU function is used here merely used as an illustrative example of an activation function and is not intended to be limiting.
  • activation functions that can be applied in the context of neural networks can include sigmoid functions, binary step functions, linear activation functions, hyperbolic functions, leaky ReLU functions, parametric ReLU functions, softmax functions, and/or swish functions, among others.
  • the input vectors and/or the weight vectors can be altered to “tune” the network.
  • a neural network can be initialized with random weights (e.g., such as analog weights). Over time, the weights can be adjusted to improve the accuracy of the neural network. This can, over time yield a neural network with high accuracy.
  • weights e.g., such as analog weights.
  • neural networks can be used for system identification and control (vehicle control, trajectory prediction, process control, natural resource management), quantum chemistry, general game playing, pattern recognition (radar systems, face identification, signal classification, 3D reconstruction, object recognition and more), sequence recognition (gesture, speech, handwritten and printed text recognition), medical diagnosis, finance (e.g. automated trading systems), data mining, visualization, machine translation, social network filtering and/or e-mail spam filtering, among others.
  • system identification and control vehicle control, trajectory prediction, process control, natural resource management
  • quantum chemistry general game playing
  • pattern recognition radar systems, face identification, signal classification, 3D reconstruction, object recognition and more
  • sequence recognition gesture, speech, handwritten and printed text recognition
  • finance e.g. automated trading systems
  • data mining visualization, machine translation, social network filtering and/or e-mail spam filtering, among others.
  • neural networks are deployed in a computing system, such as a host computing system (e.g., a desktop computer, a supercomputer, etc.) or a cloud computing environment.
  • a computing system such as a host computing system (e.g., a desktop computer, a supercomputer, etc.) or a cloud computing environment.
  • data to be subjected to the neural network as part of an operation to train the neural network can be stored in a memory resource, such as a NAND storage device, and a processing resource, such as a central processing unit, can access the data and execute instructions to process the data using the neural network.
  • a processing resource such as a central processing unit
  • Some approaches may also utilize specialized hardware such a field- programmable gate array or an application-specific integrated circuit as part of neural network training.
  • DRAM dynamic random-access memory
  • Data that may be used for performing neural network (or neuromorphic) operations can be stored in particular formats.
  • data can be stored as posits, as will be described below, in order to increase accuracy of the data.
  • performing neural network or neuromorphic operations on data stored as posits can be difficult or cumbersome.
  • Data stored in an analog format can easier to perform neural network or neuromorphic operations.
  • data can be processed in order to increase the data’s accuracy and converted to another format in order to be processed by a neural network.
  • first format e.g., a non-analog format or a posit format
  • second format e.g., an analog format
  • a non-analog format can include a format referred to as a
  • Type I unums are a superset of the IEEE 754 standard floating-point format that use a “ubif ’ at the end of the fraction to indicate whether a real number is an exact float, or if it lies in the interval between adjacent floats.
  • the sign, exponent, and fraction bits in a Type I unum take their definition from the IEEE 754 floating-point format, however, the length of the exponent and fraction fields of Type I unums can vary dramatically, from a single bit to a maximum user-definable length.
  • Type I unums can behave similar to floating-point numbers, however, the variable bit length exhibited in the exponent and fraction bits of the Type I unum can require additional management in comparison to floats.
  • bit strings e.g., strings of bits that can represent a number
  • bit strings are represented in terms of three sets of integers or sets of bits - a set of bits referred to as a “base,” a set of bits referred to as an “exponent,” and a set of bits referred to as a “mantissa” (or significand).
  • the sets of integers or bits that define the format in which a binary number string is stored may be referred to herein as an “numeric format,” or “format,” for simplicity.
  • a posit bit string may include four sets of integers or sets of bits (e.g., a sign, a regime, an exponent, and a mantissa), which may also be referred to as a “numeric format,” or “format,” (e.g., a second format).
  • two infinities e.g., + ⁇ and - ⁇
  • two kinds of “NaN” (not-a-number): a quiet NaN and a signaling NaN, may be included in a bit string.
  • Arithmetic formats can include binary and/or decimal floating-point data, which can include finite numbers, infinities, and/or special NaN values.
  • Interchange formats can include encodings (e.g., bit strings) that may be used to exchange floating-point data.
  • Rounding rules can include a set of properties that may be satisfied when rounding numbers during arithmetic operations and/or conversion operations.
  • Floating-point operations can include arithmetic operations and/or other computational operations such as trigonometric functions.
  • Exception handling can include indications of exceptional conditions, such as division by zero, overflows, etc.
  • Type II unums are generally incompatible with floats, which permits a clean, mathematical design based on projected real numbers.
  • a Type II unum can include n bits and can be described in terms of a “ «-lattice” in which quadrants of a circular projection are populated with an ordered set of 2 n ⁇ 3 - 1 real numbers.
  • the values of the Type II unum can be reflected about an axis bisecting the circular projection such that positive values he in an upper right quadrant of the circular projection, while their negative counterparts he in an upper left quadrant of the circular projection.
  • the lower half of the circular projection representing a Type II unum can include reciprocals of the values that he in the upper half of the circular projection.
  • Type II unums generally rely on a look-up table for most operations. For example, the size of the look-up table can limit the efficacy of Type II unums in some circumstances. However, Type II unums can provide improved computational functionality in comparison with floats under some conditions. [0026]
  • the Type III unum format is referred to herein as a “posit format” or, for simplicity, a “posit.” In contrast to floating-point bit strings, posits can, under certain conditions, allow for a broader dynamic range and a higher accuracy (e.g., precision) than floating-point numbers with the same bit width.
  • the use of posits in computing systems can allow for higher accuracy and/or precision than floating-point numbers, which can further improve the functioning of a computing system in comparison to some approaches (e.g., approaches which rely upon floating-point format bit strings).
  • performing neural network or neuromorphic operations on data stored in the Type III unum format can be more difficult than performing such operations on data stored in an analog format.
  • Embodiments herein are directed to hardware circuitry (e.g., control circuitry, a digital to analog converter, an analog to digital converter, etc.) configured to perform various operations on bit strings to improve the overall functioning of a computing device.
  • hardware circuitry e.g., control circuitry, a digital to analog converter, an analog to digital converter, etc.
  • embodiments herein are directed to hardware circuitry that is configured to perform conversion operations to convert a format of a bit string from a first format (e.g., a unum format, a posit format, etc.) to a second format (e.g., an analog format).
  • a first format e.g., a unum format, a posit format, etc.
  • a second format e.g., an analog format
  • the hardware circuitry can be further operated to convert the results of the operations back to the first format (e.g., to an unum format, a posit format, etc.), which can, in turn, be transferred to different circuitry (e.g., a host, a memory device, etc.) of the computing system.
  • the first format e.g., to an unum format, a posit format, etc.
  • different circuitry e.g., a host, a memory device, etc.
  • the hardware circuitry can facilitate improved performance of the neuromorphic operations for such neural network purposes while still maintaining improved accuracy and/or precision in the performance of operations performed external to the memory device and/or the neuromorphic memory array on the data in the unum or posit format, improved speed in performing the unum or posit operations, and/or a reduced required storage space for bit strings prior to, during, or subsequent to, performance of arithmetic and/or logical operations.
  • the term “resident on” refers to something that is physically located on a particular component.
  • the term “resident on” may be used interchangeably with other terms such as “deployed on” or “located on,” herein.
  • designators such as “N,” “M,” etc., particularly with respect to reference numerals in the drawings, indicate that a number of the particular feature so designated can be included. It is also to be understood that the terminology used herein is for the purpose of describing particular embodiments only, and is not intended to be limiting. As used herein, the singular forms “a,” “an,” and “the” can include both singular and plural referents, unless the context clearly dictates otherwise. In addition, “a number of,” “at least one,” and “one or more” (e.g., a number of memory banks) can refer to one or more memory banks, whereas a “plurality of’ is intended to refer to more than one of such things.
  • the words “can” and “may” are used throughout this application in a permissive sense (i.e., having the potential to, being able to), not in a mandatory sense (i.e., must).
  • the term “include,” and derivations thereof, means “including, but not limited to.”
  • the terms “coupled” and “coupling” mean to be directly or indirectly connected physically or for access to and movement (transmission) of commands and/or data, as appropriate to the context.
  • bit strings,” “data,” and “data values” are used interchangeably herein and can have the same meaning, as appropriate to the context.
  • Figure 1 is a functional block diagram in the form of a computing system 100 including an apparatus including a host 102 and a memory device 104 in accordance with a number of embodiments of the present disclosure.
  • an “apparatus” can refer to, but is not limited to, any of a variety of structures or combinations of structures, such as a circuit or circuitry, a die or dice, a module or modules, a device or devices, or a system or systems, for example.
  • the memory device 104 can include one or more memory modules (e.g., single in-line memory modules, dual in-line memory modules, etc.).
  • the memory device 104 can include volatile memory and/or non-volatile memory.
  • the memory device 104 can include a multi-chip device.
  • a multi-chip device can include a number of different memory types and/or memory modules.
  • the memory device 104 can include non-volatile or volatile memory on any type of a module.
  • the memory device 104 can provide main memory for the computing system 100 or can be used as additional memory or storage throughout the computing system 100.
  • the memory device 104 can include one or more neuromorphic memory arrays 130 (e.g., arrays of memory cells), which can include volatile and/or non-volatile memory cells.
  • the memory array 130 can be a flash array with aNAND architecture, for example. Embodiments are not limited to a particular type of memory device, however, and the memory device 104 can include RAM, ROM, DRAM, SDRAM, PCRAM, RRAM, and/or flash memory, among others.
  • the memory device 104 can include flash memory devices such as NAND or NOR flash memory devices. Embodiments are not so limited, however, and the memory device 104 can include other non-volatile memory devices such as non-volatile random-access memory devices (e.g., NVRAM, ReRAM, FeRAM, MRAM, PCM), “emerging” memory devices such as 3-D Crosspoint (3D XP) memory devices, etc., or combinations thereof.
  • a 3D XP array of non-volatile memory can perform bit storage based on a change of bulk resistance, in conjunction with a stackable cross-gridded data access array. Additionally, in contrast to many flash-based memories, 3D XP non-volatile memory can perform a write in-place operation, where a non-volatile memory cell can be programmed without the non-volatile memory cell being previously erased.
  • a host 102 can be coupled to the memory device 104.
  • the host 102 can be coupled to the memory device 104 via one or more channels 103 (e.g., buses, interfaces, communication paths, etc.).
  • the control circuitry 120 of the memory device 104 can be coupled to the memory array 130 via a channel 107.
  • the channel(s) 103 can be used to transfer data between the memory system 104 and a host 102 and can be in the form of a standardized interface.
  • the channel(s) 103 can be a serial advanced technology attachment (SATA), peripheral component interconnect express (PCIe), or a universal serial bus (USB), a double data rate (DDR) interface, among other connectors and interfaces.
  • SATA serial advanced technology attachment
  • PCIe peripheral component interconnect express
  • USB universal serial bus
  • DDR double data rate interface
  • channel (s) 103 can provide an interface for passing control, address, data, and other signals between the memory system 104 and a host 102 having compatible receptors for the channel (s) 103.
  • the host 102 can be a host system such as a personal laptop computer, a desktop computer, a digital camera, a mobile telephone, an intemet- of-things (IoT) enabled device, or a memory card reader, graphics processing unit (e.g., a video card), among various other types of hosts.
  • the host 102 can include a system motherboard and/or backplane and can include a number of memory access devices, e.g., a number of processing devices (e.g., one or more processors, microprocessors, or some other type of controlling circuitry).
  • a processor can intend one or more processors, such as a parallel processing system, a number of coprocessors, etc.
  • the system 100 can include separate integrated circuits or both the host 102, the memory device 104, and the memory array 130 can be on the same integrated circuit.
  • the system 100 can be, for instance, a server system and/or a high-performance computing (HPC) system and/or a portion thereof.
  • HPC high-performance computing
  • FIG. 1 illustrates a system having a Von Neumann architecture
  • embodiments of the present disclosure can be implemented in non-Von Neumann architectures, which may not include one or more components (e.g., CPU, ALU, etc.) often associated with a Von Neumann architecture.
  • the host 102 can be responsible for executing an operating system for a computing system 100 that includes the memory device 104. Accordingly, in some embodiments, the host 102 can be responsible for controlling operation of the memory device 104. For example, the host 102 can execute instructions (e.g., in the form of an operating system) that manage the hardware of the computing system 100 such as scheduling tasks, executing applications, controlling peripherals, etc.
  • the memory device 104 which is shown in more detail in
  • FIGS. 2A and 2B can include control circuitry 120, which can include a processing resource 122 and a memory resource 124.
  • the processing resource 122 can be provided in the form of an integrated circuit, such as an application- specific integrated circuit (ASIC), field programmable gate array (FPGA), system-on-a-chip, or other combination of hardware and/or circuitry that is configured to perform arithmetic and/or logical operations on bit string received from the host 102 and/or other external devices, as described in more detail, herein.
  • the processing resource 122 can include an arithmetic logic unit (ALU).
  • ALU arithmetic logic unit
  • the ALU can include circuitry (e.g., hardware, logic, one or more processing devices, etc.) to perform operations (e.g., arithmetic operations, logical operations, bitwise operations, etc.) such as the operations described above, on integer binary bit strings, such as bit strings in the posit format.
  • operations e.g., arithmetic operations, logical operations, bitwise operations, etc.
  • the processing resource 122 can include a state machine and/or an instruction set architecture (or combinations thereol) in addition to, or in lieu of the ALU, as described in more detail in connection with Figure 5, herein.
  • the processing resource 122 can be configured to receive one or more bit strings (e.g., a plurality of bits) in a posit format and/or cause performance of operations such as arithmetic and/or logical operations using the bit strings in the posit format.
  • bit strings e.g., a plurality of bits
  • bit strings in the floating point format which include three integers or sets of bits - a set of bits referred to as a “base,” a set of bits referred to as an “exponent,” and a set of bits referred to as a “mantissa” (or significand)
  • the bit string(s) in the posit format include four sets of bits- at least one bit referred to as a “sign,” a set of bits referred to as a “regime,” a set of bits referred to as an “exponent,” and a set of bits referred to as a “mantissa” (or significand).
  • a set of bits is intended to refer to a subset of bits included in a bit string. Examples of the sign, regime, exponent, and mantissa sets of bits are described in more detail in connection with Figures 3 and 4A-4B, herein.
  • the processing resource 122 can be configured to perform (or cause performance ol) arithmetic operations such as addition, subtraction, multiplication, division, fused multiply addition, multiply- accumulate, dot product units, greater than or less than, absolute value (e.g., FABSO), fast Fourier transforms, inverse fast Fourier transforms, sigmoid function, convolution, square root, exponent, and/or logarithm operations, and/or logical operations such as AND, OR, XOR, NOT, etc., as well as trigonometric operations such as sine, cosine, tangent, etc. using the posit bit strings.
  • absolute value e.g., FABSO
  • fast Fourier transforms inverse fast Fourier transforms
  • sigmoid function e.g., convolution, square root, exponent, and/or logarithm operations
  • convolution e.g., square root, exponent
  • logarithm operations e.g., arithmetic operations
  • logical operations such
  • processing resource 122 may be configured to perform (or cause performance of) other arithmetic and/or logical operations.
  • the control circuitry 120 can cause a resultant (e.g., a resultant bit string representing a result of the arithmetic operation and/or the logical operation) bit string to be transferred to the host 102, and/or the neuromorphic memory array 130.
  • a resultant bit string can be sent to the neuromorphic memory array 130 and the resultant bit string can be converted to an analog format.
  • the resultant bit string in the analog format can be used as an input to a number of neuromorphic operations.
  • control circuitry 120 can transfer the resultant bit string(s) to the host 102, and/or the neuromorphic memory array 130, for example, in the post format (and in the case of the neuromorphic memory array 130, converted to an analog format).
  • the control circuitry 120 can further include a memory resource
  • the memory resource 124 can include volatile memory resource, non-volatile memory resources, or a combination of volatile and non-volatile memory resources.
  • the memory resource 124 can be a random- access memory (RAM) such as static random-access memory (SRAM).
  • RAM random- access memory
  • SRAM static random-access memory
  • the memory resource 124 can be a cache, one or more registers, NVRAM, ReRAM, FeRAM, MRAM, PCM), “emerging” memory devices such as 3-D Crosspoint (3D XP) memory devices, etc., or combinations thereof.
  • the control circuitry 120 can be communicatively coupled to the neuromorphic memory array 130 via one or more channels 107.
  • the neuromorphic memory array 130 can be a DRAM array, SRAM array, STT RAM array, PCRAM array, TRAM array, RRAM array, NAND flash array, and/or NOR flash array, for instance.
  • the array 130 can comprise memory cells arranged in rows coupled by access lines, which may be referred to herein as word lines or select lines, and columns coupled by sense lines, which may be referred to herein as data lines or digit lines. Although a single array 130 is shown in Figure 1, embodiments are not so limited. For instance, memory device 104 a number of memory arrays 130 (e.g., a number of banks of DRAM cells, NAND flash cells, etc.).
  • the embodiment of Figure 1 can include additional circuitry that is not illustrated so as not to obscure embodiments of the present disclosure.
  • the memory device 104 can include address circuitry to latch address signals provided over I/O connections through I/O circuitry. Address signals can be received and decoded by a row decoder and a column decoder to access the memory device 104 and/or the neuromorphic memory array 130. It will be appreciated by those skilled in the art that the number of address input connections can depend on the density and architecture of the memory device 104 and/or the memory array 130.
  • Figure 2A is another functional block diagram in the form of a computing system including an apparatus 200 including a host 202 and a memory device 204 in accordance with a number of embodiments of the present disclosure.
  • the memory device 204 can include control circuitry 220, which can be analogous to the control circuitry 120 illustrated in Figure 1.
  • the host 202 can be analogous to the host 102 illustrated in Figure 1
  • the memory device 204 can be analogous to the memory device 104 illustrated in Figure 1
  • the neuromorphic memory array 230 can be analogous to the neuromorphic memory array 130 illustrated in Figure 1.
  • Each of the components can be separately referred to herein as an “apparatus.”
  • the host 202 can be communicatively coupled to the memory device 204 via one or more channels 203, 205.
  • the channels 203, 205 can be interfaces, buses, communication paths, or other physical connections that allow for data and/or commands to be transferred between the host 202 and the memory device 204.
  • the channels 203, 205 can be used to transfer data between the memory system 204 and a host 202 and can be in the form of a standardized interface.
  • the channels 203, 205 can be a serial advanced technology attachment (SATA), peripheral component interconnect express (PCIe), or a universal serial bus (USB), a double data rate (DDR) interface, among other connectors and interfaces.
  • SATA serial advanced technology attachment
  • PCIe peripheral component interconnect express
  • USB universal serial bus
  • DDR double data rate
  • channels 203, 205 can provide an interface for passing control, address, data, and other signals between the memory system 204 and a host 202 having compatible receptors for the channels 203, 205.
  • commands to cause initiation of an operation e.g., an operation to perform arithmetic and/or logical operations on the bit string(s) in the posit format
  • commands to cause initiation of an operation can be transferred from the host via the channels 203, 205.
  • control circuitry 220 can perform the arithmetic and/or logical operations in response to an initiation command transferred from the host 202 via one or more of the channels 203, 205 in the absence of an intervening command from the host 202. That is, once the control circuitry 220 has received the command to initiate performance of an operation from the host 202, the operations can be performed by the control circuitry 220 in the absence of additional commands from the host 202.
  • control circuitry 220 can perform the operations in response to receipt of a bit string (e.g., a bit string in the posit format) in the absence of a command (e.g., an initiation command) from the host 202 specifying that the operation is to be performed.
  • a command e.g., an initiation command
  • the control circuitry 220 can be configured to self-initiate performance of arithmetic and/or logical operations on the received bit string(s) in response to receiving the bit string(s).
  • the data in response to receiving a command to perform neuromorphic operations on the data, the data can be converted to an analog format while being sent to the neuromorphic memory array 230.
  • a “first level of precision” and a “second level of precision” generally refer to the accuracy of a bit string and/or a resultant bit string that represent a result of an operation performed using one or more bit strings.
  • floating-point format bit strings can be described herein as having a “first level of precision”
  • unum bit strings e.g., posit format bit strings
  • second level of precision because, as described in more detail herein, unum bit strings can offer a higher level of precision under certain conditions than floating-point formatted numbers.
  • the floating-point format or the unum format can refer to a digital format while an additional format can include an analog format.
  • a digital format can use discrete values such as a “0”or a “1”, while an analog format can use more continuous values and can represent physical measurements along that continuum.
  • the memory device 204 can include a register access component 206, a high speed interface (HSI) 208, a controller 210, one or more extended row address (XRA) component(s) 212, main memory input/output (I/O) circuitry 214, row address strobe (RAS)/column address strobe (CAS) chain control circuitry 216, a RAS/CAS chain component 218, control circuitry 220, and a neuromorphic memory array 230.
  • the control circuitry 220 is, as shown in Figure 2A, located in an area of the memory device 204 that is physically distinct from the neuromorphic memory array 230. That is, in some embodiments, the control circuitry 220 is located in a periphery location of the neuromorphic memory array 230.
  • the register access component 206 can facilitate transferring and fetching of data from the host 202 to the memory device 204 and from the memory device 204 to the host 202.
  • the register access component 206 can store addresses (or facilitate lookup of addresses), such as memory addresses, that correspond to data that is to be transferred to the host 202 form the memory device 204 or transferred from the host 202 to the memory device 204.
  • the register access component 206 can facilitate transferring and fetching data that is to be operated upon by the control circuitry 220 and/or the register access component 206 can facilitate transferring and fetching data that is has been operated upon by the control circuitry 220 for transfer to the host 202.
  • the HSI 208 can provide an interface between the host 202 and the memory device 204 for commands and/or data traversing the channel 205.
  • the HSI 208 can be a double data rate (DDR) interface such as a DDR3, DDR4, DDR5, etc. interface.
  • DDR double data rate
  • Embodiments are not limited to a DDR interface, however, and the HSI 208 can be a quad data rate (QDR) interface, peripheral component interconnect (PCI) interface (e.g., a peripheral component interconnect express (PCIe)), or other suitable interface for transferring commands and/or data between the host 202 and the memory device 204 via the channel(s) 203, 205.
  • PCI peripheral component interconnect express
  • the controller 210 can be responsible for executing instructions from the host 202 and accessing the control circuitry 220 and/or the neuromorphic memory array 230.
  • the controller 210 can be a state machine, a sequencer, or some other type of controller.
  • the controller 210 can receive commands from the host 202 (via the HSI 208, for example) and, based on the received commands, control operation of the control circuitry 220 and/or the neuromorphic memory array 230.
  • the controller 210 can receive a command from the host 202 to cause performance of an arithmetic and/or logical operation on received bit strings using the control circuitry 220. Responsive to receipt of such a command, the controller 210 can instruct the control circuitry 220 to begin performance of the arithmetic and/or logical operation(s).
  • the controller 210 can be a global processing controller and may provide power management functions to the memory device 204. Power management functions can include control over power consumed by the memory device 204 and/or the neuromorphic memory array 230. For example, the controller 210 can control power provided to various banks of the neuromorphic memory array 230 to control which banks of the neuromorphic memory array 230 are operational at different times during operation of the memory device 204. This can include shutting certain banks of the neuromorphic memory array 230 down while providing power to other banks of the neuromorphic memory array 230 to optimize power consumption of the memory device 230. In some embodiments, the controller 210 controlling power consumption of the memory device 204 can include controlling power to various cores of the memory device 204 and/or to the control circuitry 220, the neuromorphic memory array 230, etc.
  • the XRA component(s) 212 are intended to provide additional functionalities (e.g., peripheral amplifiers) that sense (e.g., read, store, cache) data values of memory cells in the neuromorphic memory array 230 and that are distinct from the neuromorphic memory array 230.
  • the XRA components 212 can include latches and/or registers. For example, additional latches can be included in the XRA component 212.
  • the latches of the XRA component 212 can be located on a periphery of the neuromorphic memory array 230 (e.g., on a periphery of one or more banks of memory cells) of the memory device 204.
  • the main memory input/output (I/O) circuitry 214 can facilitate transfer of data and/or commands to and from the neuromorphic memory array 230.
  • the main memory I/O circuitry 214 can facilitate transfer of bit strings, data, and/or commands from the host 202 and/or the control circuitry 220 to and from the neuromorphic memory array 230.
  • the main memory I/O circuitry 214 can include one or more direct memory access (DMA) components that can transfer the bit strings (e.g., posit bit strings stored as blocks of data) from the control circuitry 220 to the neuromorphic memory array 230, and vice versa.
  • DMA direct memory access
  • the main memory I/O circuitry 214 can facilitate transfer of bit strings, data, and/or commands from the neuromorphic memory array 230 to the control circuitry 220 so that the control circuitry 220 can perform arithmetic and/or logical operations on the bit strings. Similarly, the main memory I/O circuitry 214 can facilitate transfer of bit strings that have had one or more operations performed on them by the control circuitry 220 to the neuromorphic memory array 230.
  • the data in the unum or posit format can be operated on by the control circuitry 220 to perform arithmetic and/or logical operations while the data is stored in an array or other location other than the neuromorphic memory array 230; and the data in the analog format, while being stored in the neuromorphic memory array 230, can be used to perform neuromorphic operations.
  • the operations can include arithmetic operations performed on bit strings in a posit format, logical operations performed on bit strings in a posit format, bitwise operations performed on bit strings in a posit format, etc., and neuromorphic operations performed on bit strings in an analog format.
  • the row address strobe (RAS)/column address strobe (CAS) chain control circuitry 216 and the RAS/CAS chain component 218 can be used in conjunction with the neuromorphic memory array 230 to latch a row address and/or a column address to initiate a memory cycle.
  • the RAS/CAS chain control circuitry 216 and/or the RAS/CAS chain component 218 can resolve row and/or column addresses of the neuromorphic memory array 230 at which read and write operations associated with the neuromorphic memory array 230 are to be initiated or terminated.
  • the RAS/CAS chain control circuitry 216 and/or the RAS/CAS chain component 218 can latch and/or resolve a specific location in the neuromorphic memory array 230 to which the bit strings that have been operated upon by the control circuitry 220 to perform neuromorphic operations are to be stored.
  • the RAS/CAS chain control circuitry 216 and/or the RAS/CAS chain component 218 can latch and/or resolve a specific location in the neuromorphic memory array 230 from which bit strings are to be transferred to the control circuitry 220 prior to the control circuitry 220 performing a neuromorphic operation on the bit string(s) in an analog format.
  • the neuromorphic memory array 230 can be a DRAM array, SRAM array, STT RAM array, PCRAM array, TRAM array, RR.AM array, NAND flash array, and/or NOR flash array, for instance, although embodiments are not limited to these particular examples.
  • the neuromorphic memory array 230 can function as main memory for the computing system 200 shown in Figure 2.
  • the neuromorphic memory array 230 can be configured to store bit strings operated on by the control circuitry 220 and/or store bit strings to be transferred to the control circuitry 220.
  • the control circuitry 220 can include logic circuitry (e.g., the processing resource 122 illustrated in Figure 1) and/or memory resource(s) (e.g., the memory resource 124 illustrated in Figure 1). As described above in connection with Figure 1 and in more detail below in connection with Figure 6, the control circuitry 220 can be configured to receive one or more bit strings in the posit format and cause performance of operations such as arithmetic and/or logical operations using the one or more bit strings in the posit format and can be configured to convert the one or more bits strings in the posit format to an analog format upon receipt by the neuromorphic memory array 230.
  • logic circuitry e.g., the processing resource 122 illustrated in Figure 1
  • memory resource(s) e.g., the memory resource 124 illustrated in Figure 1
  • the control circuitry 220 can be configured to receive one or more bit strings in the posit format and cause performance of operations such as arithmetic and/or logical operations using the one or more bit strings in the posit format and can
  • bit strings (e.g., data, a plurality of bits, etc.) can be received by the control circuitry 220 from, for example, the host 202 in a first format (e.g., posit format), and/or the neuromorphic memory array 230 (e.g., in an analog format), and stored by the control circuitry 220, for example in the memory resource (e.g., the memory resource 624 illustrated in Figure 6, herein) of the control circuitry 220.
  • the control circuitry 220 can perform arithmetic and/or logical operations (or cause arithmetic and/or logical operations to be performed on) the bit string(s), as described in more detail in connection with Figure 6, herein.
  • posits can provide improved accuracy (e.g., precision) and may require less storage space (e.g., may contain a smaller number of bits) than corresponding bit strings represented in the floating-point format. Accordingly, by converting the posit bit strings to analog bit strings and performing neuromorphic operations using neural networks using the control circuitry 220, performance of the computing system 200 may be improved in comparison to approaches that utilize posit bit strings for performance of neuromorphic operations, because neuromorphic operations may be performed more quickly on the analog bit strings (e.g., because the neuromorphic operations lend themselves more efficient when working with analog data.
  • control circuitry 220 can perform (or cause performance of) arithmetic and/or logical operations on the received posit bit strings.
  • control circuitry 220 can be configured to perform (or cause performance of) arithmetic operations such as addition, subtraction, multiplication, division, fused multiply addition, multiply-accumulate, dot product units, greater than or less than, absolute value (e.g., FABS()), fast Fourier transforms, inverse fast Fourier transforms, sigmoid function, convolution, square root, exponent, and/or logarithm operations, and/or logical operations such as AND, OR, XOR, NOT, etc., as well as trigonometric operations such as sine, cosine, tangent, etc. on the received posit bit strings.
  • arithmetic operations such as addition, subtraction, multiplication, division, fused multiply addition, multiply-accumulate, dot product units, greater than or less than, absolute value (e.g., FABS()), fast Fourier transforms, inverse fast Fourier transforms, sigmoid function, convolution, square root, exponent, and/or logarithm operations, and/or logical operations
  • control circuitry 220 may be configured to perform (or cause performance of) other arithmetic and/or logical operations on posit bit strings.
  • control circuitry 220 may perform the above-listed operations in conjunction with execution of one or more machine learning algorithms.
  • the control circuitry 220 may perform operations related to one or more neural networks such as is used in the neuromorphic memory array 230.
  • Neural networks may allow for an algorithm to be trained over time to determine an output response based on input signals. For example, over time, a neural network may essentially leam to better maximize the chance of completing a particular goal. This may be advantageous in machine learning applications because the neural network may be trained over time with new data to achieve better maximization of the chance of completing the particular goal.
  • a neural network may be trained over time to improve operation of particular tasks and/or particular goals.
  • machine learning e.g., neural network training
  • may be processing intensive e.g., may consume large amounts of computer processing resources
  • time intensive e.g., may require lengthy calculations that consume multiple cycles to be performed
  • the control circuitry 220 by performing such operations using the control circuitry 220, for example, by performing such operations on bit strings in the analog format, the amount of processing resources and/or the amount of time consumed in performing the operations may be reduced in comparison to approaches in which such operations are performed using bit strings in a posit format.
  • Figure 2B is a functional block diagram in the form of a computing system 200 including a host 202, a memory device 204, an application-specific integrated circuit 223, and a field programmable gate array 221 in accordance with a number of embodiments of the present disclosure.
  • each of the components can be separately referred to herein as an “apparatus.”
  • the host 202 can be coupled to the memory device 204 via channel(s) 203, which can be analogous to the channel(s) 103 illustrated in Figure 1.
  • the field programmable gate array (FPGA) 221 can be coupled to the host 202 via channel(s) 217 and the application-specific integrated circuit (ASIC) 223 can be coupled to the host 202 via channel(s) 219.
  • FPGA field programmable gate array
  • ASIC application-specific integrated circuit
  • the channel(s) 217 and/or the channel(s) 219 can include a peripheral serial interconnect express (PCIe) interface, however, embodiments are not so limited, and the channel(s) 217 and/or the channel(s) 219 can include other types of interfaces, buses, communication channels, etc. to facilitate transfer of data between the host 202 and the FPGA 221 and/or the ASIC 223.
  • PCIe peripheral serial interconnect express
  • arithmetic and/or logical operations that can be performed by the FPGA 221 and/or the ASIC 223 include arithmetic operations such as addition, subtraction, multiplication, division, fused multiply addition, multiply-accumulate, dot product units, greater than or less than, absolute value (e.g., FABS()), fast Fourier transforms, inverse fast Fourier transforms, sigmoid function, convolution, square root, exponent, and/or logarithm operations, and/or logical operations such as AND, OR, XOR, NOT, etc., as well as trigonometric operations such as sine, cosine, tangent, etc. using the posit bit strings.
  • arithmetic operations such as addition, subtraction, multiplication, division, fused multiply addition, multiply-accumulate, dot product units, greater than or less than, absolute value (e.g., FABS()), fast Fourier transforms, inverse fast Fourier transforms, sigmoid function, convolution, square root, exponent, and/
  • the FPGA 221 can include a state machine 227 and/or register(s)
  • the state machine 227 can include one or more processing devices that are configured to perform operations on an input and produce an output.
  • the FPGA 221 can be configured to receive posit bit strings from the host 202 and perform arithmetic and/or logical operations on the posit bit strings to produce resultant posit bit strings that represents a result of the operation performed on the received posit bit strings.
  • the FPGA 221 can be configured to cause performance of neuromorphic operations in the neuromorphic memory array 230 on data that has been converted to an analog format.
  • the register(s) 229 of the FPGA 221 can be configured to buffer and/or store the posit bit strings received form the host 202 prior to the state machine 227 performing an operation on the received posit bit strings.
  • the register(s) 229 of the FPGA 221 can be configured to buffer and/or store a resultant posit bit string that represents a result of the operation performed on the received posit bit strings prior to transferring the result to circuitry external to the ASIC 233, such as the host 202 or the memory device 204, etc.
  • the ASIC 223 can include logic 241 and/or a cache 243.
  • the logic 241 can include circuitry configured to perform operations on an input and produce an output.
  • the ASIC 223 is configured to receive posit bit strings from the host 202 and perform arithmetic and/or logical operations on the posit bit strings to produce resultant posit bit strings that represents a result of the operation performed on the received posit bit strings. Likewise, the ASIC 223 can facilitate the conversion of the posit bit strings to analog bit strings and the subsequent neuromorphic operations to be performed on the analog bit strings.
  • the cache 243 of the ASIC 223 can be configured to buffer and/or store the posit bit strings received form the host 202 prior to the logic 241 performing an operation on the received posit bit strings.
  • the cache 243 of the ASIC 223 can be configured to buffer and/or store a resultant posit bit string that represents a result of the operation performed on the received posit bit strings prior to transferring the result to circuitry external to the ASIC 233, such as the host 202 or the memory device 204, etc.
  • FPGA 227 is shown as including a state machine
  • the FPGA 221 can include logic, such as the logic 241, and/or a cache, such as the cache 243 in addition to, or in lieu of, the state machine 227 and/or the register(s) 229.
  • the ASIC 223 can, in some embodiments, include a state machine, such as the state machine 227, and/or register(s), such as the register(s) 229 in addition to, or in lieu of, the logic 241 and/or the cache 243.
  • Figure 3 is an example of an «-bit universal number, or “unum” with es exponent bits.
  • the «-bit unum is a posit bit string 331.
  • the «-bit posit 331 can include a set of sign bit(s) (e.g., a sign bit 333), a set of regime bits (e.g., the regime bits 335), a set of exponent bits (e.g., the exponent bits 337), and a set of mantissa bits (e.g., the mantissa bits 339).
  • the mantissa bits 339 can be referred to in the alternative as a “fraction portion” or as “fraction bits,” and can represent a portion of a bit string (e.g., a number) that follows a decimal point.
  • the sign bit 333 can be zero (0) for positive numbers and one (1) for negative numbers.
  • the regime bits 335 are described in connection with Table 1, below, which shows (binary) bit strings and their related numerical meaning, k.
  • Table 1 the numerical meaning, k, is determined by the run length of the bit string.
  • the letter x in the binary portion of Table 1 indicates that the bit value is irrelevant for determination of the regime, because the (binary) bit string is terminated in response to successive bit flips or when the end of the bit string is reached.
  • the bit string terminates in response to a zero flipping to a one and then back to a zero. Accordingly, the last zero is irrelevant with respect to the regime and all that is considered for the regime are the leading identical bits and the first opposite bit that terminates the bit string (if the bit string includes such bits).
  • the regime bits 335 r correspond to identical bits in the bit string, while the regime bits 335 r correspond to an opposite bit that terminates the bit string.
  • the regime bits r correspond to the first two leading zeros, while the regime bit(s) r correspond to the one.
  • the final bit corresponding to the numerical k which is represented by the X in Table 1 is irrelevant to the regime.
  • the exponent bits 337 correspond to an exponent e, as an unsigned number. In contrast to floating-point numbers, the exponent bits 337 described herein may not have a bias associated therewith. As a result, the exponent bits 337 described herein may represent a scaling by a factor of 2 e . As shown in Figure 3, there can be up to es exponent bits (ei, ei. e , . . ., e es ), depending on how many bits remain to right of the regime bits 335 of the «-bit posit 331.
  • this can allow for tapered accuracy of the «-bit posit 331 in which numbers which are nearer in magnitude to one have a higher accuracy than numbers which are very large or very small.
  • the tapered accuracy behavior of the «-bit posit 331 shown in Figure 3 may be desirable in a wide range of situations.
  • the mantissa bits 339 represent any additional bits that may be part of the «-bit posit 331 that lie to the right of the exponent bits 337. Similar to floating-point bit strings, the mantissa bits 339 represent a fraction which can be analogous to the fraction 1/ where /includes one or more bits to the right of the decimal point following the one. In contrast to floating-point bit strings, however, in the «-bit posit 331 shown in Figure 3, the “hidden bit” (e.g., the one) may always be one (e.g., unity), whereas floating point bit strings may include a subnormal number with a “hidden bit” of zero (e.g., 0.f).
  • the “hidden bit” e.g., the one
  • floating point bit strings may include a subnormal number with a “hidden bit” of zero (e.g., 0.f).
  • Figure 4A is an example of positive values for a 3-bit posit 431.
  • the precision of a posit 431 can be increased by appending bits the bit string, as shown in Figure 4B.
  • appending a bit with a value of one (1) to bit strings of the posit 431-1 increases the accuracy of the posit 431 as shown by the posit 431-2 in Figure 4B.
  • appending a bit with a value of one to bit strings of the posit 431-2 in Figure 4B increases the accuracy of the posit 431-2 as shown by the posit 431-3 shown in Figure 4B.
  • maxpos may be equivalent to useed and minpos may be equivalent to Between maxpos and ⁇ , a new bit value may be maxpos * useed, and between zero and minpos, a new bit value may be .
  • Figure 4B is an example of posit construction using two exponent bits.
  • Figure 4B only the right half of projective real numbers, however, it will be appreciated that negative projective real numbers that correspond to their positive counterparts shown in Figure 4B can exist on a curve representing a transformation about they- ax is of the curves shown in Figure 4B.
  • the posits 431-1, 431-2, 431-3 shown in Figure 4B each include only two exception values: Zero (0) when all the bits of the bit string are zero and ⁇ when the bit string is a one (1) followed by all zeros. It is noted that the numerical values of the posits 431-1, 431-2, 431-3 shown in Figure 4 are exactly useed k .
  • the numerical values of the posits 431-1, 431-2, 431-3 shown in Figure 4 are exactly useed to the power of the k value represented by the regime (e.g., the regime bits 335 described above in connection with Figure 3).
  • the posit 431-1 has 256
  • the corresponding bit strings have an additional exponent bit appended thereto.
  • the numerical values 1/16, 1 ⁇ 4, 1, and 4 will have an exponent bit appended thereto. That is, the final one corresponding to the numerical value 4 is an exponent bit, the final zero corresponding o the numerical value 1 is an exponent bit, etc.
  • the posit 431-3 is a 5-bit posit generated according to the rules above from the 4-bit posit 431-2. If another bit was added to the posit 431-3 in Figure 4B to generate a 6-bit posit, mantissa bits 339 would be appended to the numerical values between 1/16 and 16.
  • bit string corresponding to a posit p is an unsigned integer ranging from — 2 n_1 to 2 n_1
  • k is an integer corresponding to the regime bits 335
  • e is an unsigned integer corresponding to the exponent bits 337.
  • the set of mantissa bits 339 is represented as (/1 /2 . . . f ⁇ and / is a value represented by l.fifi . . . fo (e.g.. by a one followed by a decimal point followed by the mantissa bits 339)
  • the p can be given by Equation 1, below.
  • the regime bits 335 have a run of three consecutive zeros corresponding to a value of -3 (as described above in connection with Table 1).
  • the scale factor contributed by the regime bits 335 is 256 -3 (e.g., useed k ).
  • the mantissa bits 339 which are given in Table 3 as 11011101, represent two-hundred and twenty-one (221) as
  • FIG. 5 is a functional block diagram 550 in the form of a neuromorphic memory array 530, a digital to analog converter 557, and an analog to digital data converter 561 in accordance with a number of embodiments of the present disclosure.
  • the neuromorphic memory array 530 can received data from a host (e.g., host 102, 202 in Figures 1, 2 A, 2B) or other external device.
  • the data can include bit strings in a first format (e.g., an unum number or posit format).
  • the data in the posit format can be converted, by a digital to analog converter 557 (e.g., where data is received and controlled (“data/ctrl”), upon receipt of the data.
  • the data can be stored, once it is converted to analog, in a plurality of analog memory cells (“AMC”) 552-1 to 552-P (hereinafter referred to as AMCs 552).
  • Address logic 554 can be used to identify a location of a particular cell, such as a first AMC 552-1, in order to store data at its correct location in the neuromorphic memory array 530.
  • neuromorphic operations can be performed using the AMCs 552 as a neural network.
  • Some neuromorphic systems may use resistive RAM (RRAM) such as PCM devices or self-selecting memory devices to perform neuromorphic operations by storing a value (or weight) of a synapse (e.g., synaptic weight).
  • RRAM resistive RAM
  • Such variable resistance memory may include memory cells configured to store multiple levels and/or that may have wide sense windows.
  • types of memory may be configured to perform at least a portion of a neuromorphic operation by performing training operations by pulse (e.g., spike) control.
  • Such training operations may include spike-timing- dependent plasticity (STDP).
  • STDP may be a form of Hebbian learning that is induced by correlation between spikes transmitted between nodes (e.g., neurons).
  • STDP may be an example of a process that adjusts the strength of connections between nodes (e.g., neurons).
  • a synaptic weight refers to the strength or amplitude of a connection between two nodes (e.g., neurons).
  • the nature and content of information transmitted through a neural network may be based in part on the properties of connections, which represent synapses, formed between the nodes.
  • the property of the connections can be synaptic weights.
  • Neuromorphic systems and devices may be designed to achieve results that may not be possible with traditional computer architectures. For example, neuromorphic systems may be used to achieve results more commonly associated with biological systems such as learning, vision or visual processing, auditory processing, advanced computing, or other processes, or a combination thereof.
  • performance of neuromorphic operations can include using the synaptic weight and/or connections between at least two memory cells to represent a synapse, or a strength or degree of connectivity of the synapse and be associated with a respective short-term connection or a long-term connection, which corresponds to the biological occurrence of short-term and long-term memory.
  • a series of neural network or neuromorphic operations can be performed in order to increase the synaptic weight between the at least two memory cells in a short-term or a long-term fashion, depending on which type of memory cell is used, as will be described below.
  • a learning event of a neural network or neuromorphic operation can represent causal propagation of spikes among neurons, enabling a weight increase for the connecting synapses.
  • a weight increase of a synapse can be represented by an increase in conductivity of a memory cell.
  • a variable resistance memory array (for example a 3D cross-point or self-selecting memory (SSM) array) can mimic an array of synapses, each characterized by a weight, or a memory cell conductance. The greater the conductance, the greater the synaptic weight and the higher the degree of memory learning.
  • Short-term memory learning can be a fast and/or reversible memory learning where the analog weight of a synapse is enhanced, namely, its electrical conduction is increased by a reversible mechanism.
  • Long-term memory learning can be a slow and/or irreversible memory learning where the cell conductance is irreversibly increased for a particular state (e.g., SET or RESET), leading to obsolete memory coming from a longer, experience-dependent learning.
  • Neuromorphic operations can be used to mimic neuro-biological architectures that may be present in a nervous system and/or to store synaptic weights associated with long-term and short-term learning or relationships are described herein.
  • a memory apparatus may include a memory array including a first portion and a second portion.
  • the first portion of the memory array can include a first plurality of variable resistance memory cells and the second portion can include a second plurality of variable resistance memory cells.
  • the second portion can be degraded through forced write cycling.
  • the degradation mechanism can include damage to the chalcogenide material.
  • the degradation mechanism can include the thermal relationship between memory cells, control via control gate coupling between memory cells, a charge loss corresponding to the memory cells, a temperature induced loss of signal or threshold, etc.
  • These neuromorphic operations can be performed on data received by the neuromorphic memory array 530.
  • the neural network can receive a large amount of data (what may be referred to as analog weights 559) used to train the neural network (e.g., the data stored in the AMCs 552).
  • analog weights 559 can be added to the data values in order to train the data values for subsequent neuromorphic processing.
  • the training can include a large volume of data that allows the neural network of the neuromorphic memory array 530 to correctly interpret the data and provide a desired or helpful result. Using the neural network processing described above, the large amount of data can train the neural network of the neuromorphic memory array 530 to detect the event or pattern and become more effective and efficient at doing so.
  • the resultant analog data can be converted, by an analog to digital converter (“Analog to digital data”) 561, to data in anon-analog format (e.g., an unum or posit format).
  • an operation (“Analog MAC + threshold”) 563 can be performed on the data in order to properly sort the analog values for conversion to the non-analog format.
  • a number of thresholds can represent which values of the 6 bits are represented by which thresholds.
  • the threshold data of the operation 563 can be used to determine how to interpret the analog data and where to locate the analog values along the corresponding scale of the non-analog format in relation to the more discrete format.
  • the resultant value of the neuromorphic operation can be sent outside the neuromorphic memory array 530 for further processing in the non analog format by control circuitry (such as control circuitry 120/220 in Figures 1 and 2A) and other storage locations. In this way, multiple levels of processing can occur on the same portion of data. For example, a command can be sent from a host to perform both arithmetic/logical operations and neuromorphic operations on a set of data.
  • the arithmetic operations can be performed while the data is in the non-analog format and outside the neuromorphic memory array 530.
  • the result of the arithmetic operation can be sent to the neuromorphic memory array 530 and converted to an analog format.
  • the data can be converted back to the non-analog format and sent back to the host with both operations being performed.
  • the data my be received in the non-analog format and converted to an analog format first in order for the neuromorphic operations to be performed first.
  • the result of the neuromorphic operation can be converted to the non-analog format and subsequent arithmetic operations can be performed.
  • FIG. 6 is a functional block diagram in the form of an apparatus 600 including control circuitry 620 in accordance with a number of embodiments of the present disclosure.
  • the control circuitry 620 can include logic circuitry 622 and a memory resource 624, which can be analogous to the processing resource 122 and the memory resource 124 illustrated in Figure 1, herein.
  • the logic circuitry 622 and/or the memory resource 624 can separately be considered an “apparatus.”
  • the control circuitry 620 can be configured to receive a bit string in the posit format from a host (e.g., the host 102/202 illustrated in Figures 1,
  • the posit bit string can be stored in the memory resource 624.
  • arithmetic and/or logical operations can be performed on the posit bit string in the absence of intervening commands from the host and/or the controller.
  • the control circuitry 620 can include sufficient processing resources and/or instructions to perform arithmetic and/or logical operations on the bit strings stored in the memory resource 624 without receiving additional commands from circuitry external to the control circuitry 620.
  • the logic circuitry 622 can be an arithmetic logic unit (ALU), a state machine, sequencer, controller, an instruction set architecture (ISA), or other type of control circuitry.
  • ALU arithmetic logic unit
  • ISA instruction set architecture
  • an ALU can include circuitry to perform operations (e.g., arithmetic operations, logical operations, bitwise operations, etc.) such as the operations described above, on integer binary numbers, such as bit strings in the posit format.
  • An instruction set architecture (ISA) can include a reduced instruction set computing (RISC) device.
  • the RISC device can include a processing resource that can employ an instruction set architecture (ISA) such as a RISC-V ISA, however, embodiments are not limited to RISC-V ISAs and other processing devices and/or ISAs can be used.
  • ISA instruction set architecture
  • the logic circuitry 622 can be configured to execute instructions (e.g., instructions stored in the INSTR 625 portion of the memory resource 624) to perform the operations above.
  • the logic circuitry 624 is provisioned with sufficient processing resources to cause performance of arithmetic and/or logical operations on the data (e.g., on bit strings) received by the control circuitry 620.
  • the resultant bit strings can be stored in the memory resource 624 and/or a memory array (e.g., the neuromorphic memory array 230 illustrated in Figure 2A, herein).
  • the stored resultant bit strings can be addressed such that they are accessible for performance of the operations.
  • the bit strings can be stored in the memory resource 624 and/or the memory array at particular physical addresses (which may have corresponding logical addresses corresponding thereto) such that the bit strings can be accessed in performing the operations.
  • the memory resource 624 can, in some embodiments, be a memory resource such as random-access memory (e.g., RAM, SRAM, etc.). Embodiments are not so limited, however, and the memory resource 624 can include various registers, caches, buffers, and/or memory arrays (e.g., 1T1C, 2T2C, 3T, etc. DRAM arrays).
  • the memory resource 624 can be configured to receive a bit string (e.g., a bit string in the posit format) from, for example, a host such as the host 102/202 illustrated in Figures 1, 2A, and/or a memory array such as the neuromorphic memory array 230 illustrated in Figures 2A, herein.
  • the memory resource 638 can have a size of approximately 256 kilobytes (KB), however, embodiments are not limited to this particular size, and the memory resource 624 can have a size greater than, or less than, 256 KB.
  • the memory resource 624 can be partitioned into one or more addressable memory regions. As shown in Figure 6, the memory resource 624 can be partitioned into addressable memory regions so that various types of data can be stored therein. For example, one or more memory regions can store instructions (“INSTR”) 625 used by the memory resource 624, one or more memory regions can store data 626-1, . .
  • LOCAL MEM. local memory
  • bit string(s) can be retrieved from the host and/or memory array in response to messages and/or commands generated by the host, a controller (e.g., the controller 210 illustrated in Figure 2A, herein), or the logic circuitry 622.
  • the commands and/or messages can be processed by the logic circuitry 622.
  • the logic circuitry 622 Once the bit string(s) are received by the control circuitry 620 and stored in the memory resource 624, they can be processed by the logic circuitry 622. Processing the bit string(s) by the logic circuitry 622 can include performing arithmetic operations and/or logical operations on the received bit string(s).
  • an analog bit string can provide neural network training results faster than the 8-bit posit bit string.
  • FIG. 7 is a flow diagram representing an example method 770 for neuromorphic operations on posits in accordance with a number of embodiments of the present disclosure.
  • the method 770 can include receiving, at a memory device, data comprising a bit string in a first format that supports arithmetic operations to a particular level of precision.
  • the memory device can be analogous to the memory device 104 illustrated in Figure 1 and 204 illustrated in Figure 2A.
  • the first format can be the posit format [00112]
  • the method 770 can include converting, via a digital to analog converter, the data to a second format that is an analog format.
  • the digital to analog converter can be analogous to the digital to analog converter 557 in Figure 5 herein.
  • the first format can be an analog format and the second format can be a posit format.
  • the second format can include a mantissa, a regime, a sign, and an exponent.
  • the method 770 can include writing the data in the second format in a memory cell of the memory device.
  • the memory cell can be a memory cell 552 as illustrated in Figure 5 herein.
  • the method 770 can further include adding an analog weight to the data in the second format resulting in adjusted data and writing the adjusted data to the memory cell.
  • the method 770 can include performing an operation using the data written to the memory cell resulting in additional data in the second format.
  • the method 770 can further include converting the additional data in the second format to additional data in the first format.
  • the method 770 can further include sending the additional data in the first format to a device external to the memory device.
  • the device external to the memory device can be a host.
  • performance of the operation can include performing a neuromorphic operation on the data written to the memory cell.
  • performance of the operation on the data in the second format occurs in a shorter amount of time or in a fewer number of sub-operations than if the operation is performed on the same data represented in the first format.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Biophysics (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Neurology (AREA)
  • Computational Linguistics (AREA)
  • Computer Hardware Design (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Memory System (AREA)
  • Advance Control (AREA)
  • Microelectronics & Electronic Packaging (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Abstract

L'invention concerne des systèmes, des appareils et des procédés relatifs à un neurone construit avec des positons. Un système donné à titre d'exemple peut comprendre un dispositif de mémoire et le dispositif de mémoire peut comprendre une pluralité de cellules de mémoire. La pluralité de cellules de mémoire peut stocker des données comprenant une chaîne de bits dans un format analogique. Une opération neuromorphique peut être effectuée sur les données dans le format analogique. Le système donné à titre d'exemple peut comprendre un convertisseur analogique-numérique couplé au dispositif de mémoire. Le convertisseur analogique-numérique peut convertir la chaîne de bits dans le format analogique stocké dans au moins l'une de la pluralité de cellules de mémoire en un format qui prend en charge des opérations arithmétiques à un niveau particulier de précision.
PCT/US2021/035125 2020-06-29 2021-06-01 Opérations neuromorphiques utilisant des positons Ceased WO2022005673A1 (fr)

Priority Applications (3)

Application Number Priority Date Filing Date Title
EP21833816.8A EP4172875A4 (fr) 2020-06-29 2021-06-01 Opérations neuromorphiques utilisant des positons
KR1020237002393A KR102813842B1 (ko) 2020-06-29 2021-06-01 포지트를 사용하는 뉴로모픽 연산
CN202180035602.9A CN115668224B (zh) 2020-06-29 2021-06-01 使用posit的神经形态运算

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US16/915,039 2020-06-29
US16/915,039 US11636323B2 (en) 2020-06-29 2020-06-29 Neuromorphic operations using posits
US16/997,599 US12159219B2 (en) 2020-08-19 2020-08-19 Neuron using posits
US16/997,599 2020-08-19

Publications (1)

Publication Number Publication Date
WO2022005673A1 true WO2022005673A1 (fr) 2022-01-06

Family

ID=79314886

Family Applications (2)

Application Number Title Priority Date Filing Date
PCT/US2021/035125 Ceased WO2022005673A1 (fr) 2020-06-29 2021-06-01 Opérations neuromorphiques utilisant des positons
PCT/US2021/039326 Ceased WO2022005944A1 (fr) 2020-06-29 2021-06-28 Neurone utilisant des postulats

Family Applications After (1)

Application Number Title Priority Date Filing Date
PCT/US2021/039326 Ceased WO2022005944A1 (fr) 2020-06-29 2021-06-28 Neurone utilisant des postulats

Country Status (4)

Country Link
EP (2) EP4172875A4 (fr)
KR (2) KR102813842B1 (fr)
CN (2) CN115668224B (fr)
WO (2) WO2022005673A1 (fr)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12079733B2 (en) * 2020-06-23 2024-09-03 Sandisk Technologies Llc Multi-precision digital compute-in-memory deep neural network engine for flexible and energy efficient inferencing
CN116432711B (zh) * 2023-02-13 2023-12-05 杭州菲数科技有限公司 SiLU激活函数的硬件实现方法、装置及计算设备
KR102840183B1 (ko) * 2024-03-21 2025-08-01 경희대학교 산학협력단 인공 신경망을 위한 데이터 포맷을 처리하는 인공 신경망 연산 장치 및 인공신경망 연산 방법
CN120670371B (zh) * 2025-08-14 2025-11-04 西安电子科技大学杭州研究院 一种基于posit格式的存算一体处理器及处理方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130144821A1 (en) * 2011-12-05 2013-06-06 Commissariat A L'energie Atomique Et Aux Energies Alternatives Digital-to-Analogue Converter and Neuromorphic Circuit Using Such a Converter
US20170103302A1 (en) * 2015-10-08 2017-04-13 Via Alliance Semiconductor Co., Ltd. Neural network unit with neural processing units dynamically configurable to process multiple data sizes
KR20180111156A (ko) * 2017-03-31 2018-10-11 에스케이하이닉스 주식회사 피드-백 라인을 가진 포스트-시냅틱 뉴런을 포함하는 뉴로모픽 소자 및 뉴로모픽 소자의 동작 방법
KR101965850B1 (ko) * 2017-04-26 2019-04-05 광주과학기술원 인공 신경망에 사용되는 활성 함수의 확률적 구현 방법 및 그를 포함하는 시스템
US20190279079A1 (en) * 2018-03-08 2019-09-12 Postech Academy-Industry Foundation Neuromorphic system with transposable memory and virtual look-up table

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6795117B2 (en) * 2001-11-06 2004-09-21 Candela Microsystems, Inc. CMOS image sensor with noise cancellation
US9830999B2 (en) * 2014-06-05 2017-11-28 Micron Technology, Inc. Comparison operations in memory
US10169701B2 (en) * 2015-05-26 2019-01-01 International Business Machines Corporation Neuron peripheral circuits for neuromorphic synaptic memory array based on neuron models
US10885429B2 (en) * 2015-07-06 2021-01-05 University Of Dayton On-chip training of memristor crossbar neuromorphic processing systems
US10360971B1 (en) 2015-11-02 2019-07-23 Green Mountain Semiconductor, Inc. Artificial neural network functionality within dynamic random-access memory
US10180808B2 (en) 2016-10-27 2019-01-15 Samsung Electronics Co., Ltd. Software stack and programming for DPU operations
US11222260B2 (en) * 2017-03-22 2022-01-11 Micron Technology, Inc. Apparatuses and methods for operating neural networks
US11354568B2 (en) * 2017-06-30 2022-06-07 Intel Corporation In-memory spiking neural networks for memory array architectures
US10956811B2 (en) * 2017-07-31 2021-03-23 Intel Corporation Variable epoch spike train filtering
JP2019057053A (ja) * 2017-09-20 2019-04-11 東芝メモリ株式会社 半導体記憶装置
KR102675781B1 (ko) * 2018-06-27 2024-06-18 삼성전자주식회사 뉴로모픽 프로세서 및 그것의 동작 방법
CN111310909B (zh) * 2020-02-24 2024-02-13 南京大学 一种浮点数转换电路

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130144821A1 (en) * 2011-12-05 2013-06-06 Commissariat A L'energie Atomique Et Aux Energies Alternatives Digital-to-Analogue Converter and Neuromorphic Circuit Using Such a Converter
US20170103302A1 (en) * 2015-10-08 2017-04-13 Via Alliance Semiconductor Co., Ltd. Neural network unit with neural processing units dynamically configurable to process multiple data sizes
KR20180111156A (ko) * 2017-03-31 2018-10-11 에스케이하이닉스 주식회사 피드-백 라인을 가진 포스트-시냅틱 뉴런을 포함하는 뉴로모픽 소자 및 뉴로모픽 소자의 동작 방법
KR101965850B1 (ko) * 2017-04-26 2019-04-05 광주과학기술원 인공 신경망에 사용되는 활성 함수의 확률적 구현 방법 및 그를 포함하는 시스템
US20190279079A1 (en) * 2018-03-08 2019-09-12 Postech Academy-Industry Foundation Neuromorphic system with transposable memory and virtual look-up table

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
CHI PING ET AL.: "2013 21st International conference on Program Comprehension (ICPC); [International Symposium on Computer Architecture (ISCA", 18 June 2016, IEEE, article "PRIME: A Novel Processing-in-Memory Architecture for Neural Network Computation in ReRAM-Based Main Memory", pages: 27 - 39
See also references of EP4172875A4
ZACHARIAH CARMICHAEL ET AL.: "Arxiv.org", 5 December 2018, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNEL UNIVERSITY, article "Deep Positron: A Deep Neural Network Using the Posit Number System"

Also Published As

Publication number Publication date
EP4172875A1 (fr) 2023-05-03
KR102813842B1 (ko) 2025-05-28
KR20230021140A (ko) 2023-02-13
EP4172874A4 (fr) 2024-07-17
WO2022005944A1 (fr) 2022-01-06
EP4172875A4 (fr) 2024-08-14
CN115668224A (zh) 2023-01-31
CN115668224B (zh) 2024-06-11
KR102813843B1 (ko) 2025-05-28
EP4172874A1 (fr) 2023-05-03
KR20230027250A (ko) 2023-02-27
CN115516463A (zh) 2022-12-23

Similar Documents

Publication Publication Date Title
KR102813842B1 (ko) 포지트를 사용하는 뉴로모픽 연산
US12079589B2 (en) Memory array data structure for posit operations
US11714605B2 (en) Acceleration circuitry
US11496149B2 (en) Bit string conversion invoking bit strings having a particular data pattern
US20250086446A1 (en) Neuron using posits
US12112258B2 (en) Neuromorphic operations using posits
US11829301B2 (en) Acceleration circuitry for posit operations
US11829755B2 (en) Acceleration circuitry for posit operations
US11782711B2 (en) Dynamic precision bit string accumulation
US11080017B2 (en) Host-based bit string conversion
US20220215235A1 (en) Memory system to train neural networks
US20200293289A1 (en) Bit string conversion
WO2020247077A1 (fr) Accumulation de chaînes de bits dans la périphérie d'un réseau de mémoire

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21833816

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2021833816

Country of ref document: EP

Effective date: 20230130