KR20190081054A - Kernel Hardware Device - Google Patents

Abstract

The present invention provides a parallel kernel hardware system and a 3D kernel hardware system. According to the present invention, the kernel hardware system comprises: a plurality of input terminals; a plurality of output terminals; first conductive lines electrically connected to the input terminals and disposed parallel to each other; and second conductive lines electrically connected to the output terminals and crossing the first conductive lines. Input data is divided into reception regions having an n x n size and the input terminals corresponding to elements of one reception region are defined as one input terminal group. The first conductive lines electrically connected to the input terminal group are electrically connected to one second conductive line through a node group and each node forming the node group is a resistance change memory. Accordingly, a plurality of input terminals receive more extended reception regions than the reception region as input data, thereby minimizing serial scanning operation. Moreover, the area of the kernel hardware system is reduced, thereby realizing high integration.

Description

Kernel Hardware Device < RTI ID = 0.0 >

The present invention relates to a kernel hardware device implementing a kernel of a Convolutional Neural Network in hardware, and more particularly, to a kernel hardware device having an extended accepting area.

Computers with Von Neumann architecture have evolved along with improvements in the performance of electronic systems based on CMOS technology. A computer with a von Neumann structure is operated through a CPU and stored in a memory, and data necessary for calculation is read from the memory to the CPU. Therefore, although it is suitable for an operation requiring an accuracy such as a numerical calculation, the efficiency is low in an operation for processing a large amount of data in parallel. In recent years, a neural network computation algorithm, which is based on the structure of a human brain neural network, has attracted attention, and a computer system of a new structure capable of processing large-scale data in parallel, such as a human brain neural network, is required.

The artificial neural network computation algorithm consists of neuron circuits and synapses. A synapse is a connection between each neuron circuit, and each synapse has a weight according to importance. The information input to the artificial neural network computation algorithm is activated by the neuron circuit through the synapse, and the activated information is transmitted to the next neuron circuit through the synapse again. Therefore, the information input to the artificial neural network calculation algorithm leads to the final result in the last neuron circuit through a plurality of neuron circuits.

Convolution neural network (CNN) is one of the computational algorithms for computer neural networks. The composite neural network includes a plurality of convolution layers and a fully connected neural network. The composite product layer extracts the features of the input image and provides a feature map that can be effectively input to a fully connected neural network.

For example, in the case of a color image consisting of 32 pixels in width and 32 pixels in length, the input image has a size of 32 × 32 × 3 (RGB three channels). For example, if a neuron of the next layer with a size of 1500, for example, a neural network, is connected to all components of the input layer, it has 32 × 32 × 3 × 1,500 = 4,608,000 connections and weights. Therefore, to prevent overload caused by unnecessary connections, the composite neural network links neurons of one layer with a small area (receptive field) rather than all the neurons of the previous layer.

One composite product layer may have a kernel made up of weights applied to each neuron in the coverage area. According to the stride, the acceptance area scans the input image, multiplies the weight value stored in the kernel, and forms a feature map that is output.

A hardware structure for efficiently processing such a composite neural network is required. IEEE Electron Device Letters 37.7 (2016): 870-873.) Discloses a method for constructing a two-dimensional kernel circuit, And a technique of scanning an input image by moving an accommodation area using an external control circuit is disclosed.

However, the process of scanning the input image may slow down the overall convolution operation. In particular, a piecewise convolutional neural network can have several successive composite product layers, and if each layer requires such a scanning process, the entire computation process can require an excessively long time. Further, a space for constructing an external control circuit for moving the accommodating area is required, which makes it impossible to downsize the circuit.

The present invention provides a kernel hardware device having an extended coverage area to solve the above-described problems.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a kernel hardware device having an extended coverage area.

According to an aspect of the present invention, there is provided an input / output device including a plurality of input terminals corresponding to elements of input data, a plurality of output terminals respectively corresponding to elements of output data composed of a matrix, And second conductive lines electrically connected to the output terminals and intersecting the first conductive lines, wherein the input data comprises at least one of a first conductivity line Wherein input terminals corresponding to the elements of one reception area are defined as one input terminal group and the first conductive lines electrically connected to the input terminal group are connected to one of the second conductive line and the node group And each node constituting the node group is electrically connected to a kernel hard disk It provides a control system.

According to an embodiment of the present invention, each node constituting one node group has a resistance value corresponding to an element value constituting the kernel matrix, and all the node groups may have the same resistance arrangement.

One embodiment of the present invention provides a parallel kernel hardware system that eliminates unnecessary conductive lines. Wherein the plurality of input terminals are disposed on a first straight line spaced apart from each other by a predetermined distance and the plurality of output terminals are disposed on a second straight line spaced apart from each other by a predetermined distance and parallel to a first straight line, Each of the first conductive lines extending from the input end to only a region where the nodes are formed and each of the second conductive lines starting from the output end and extending only to a region where the nodes are formed, The conductive lines may be orthogonal to each other.

Resistance change layers are formed between the first conductive lines and the second conductive lines at points where the first conductive lines and the second conductive lines intersect up and down, A conductive path is selectively formed only in the change layer, and the conductivity of the conductive path may correspond to an element value of the kernel matrix.

One embodiment of the present invention provides a three-dimensional kernel hardware system that reduces the required area through three-dimensional stacking. In a three-dimensional kernel hardware system, the first conductive lines are spaced apart from each other and arranged perpendicular to the substrate, the second conductive lines being spaced apart from each other and stacked on the substrate in parallel, And the second conductive lines may be orthogonal to each other.

Wherein the second conductive lines have an area and the first conductive lines are arranged in a matrix through the second conductive lines, and the resistance-variable layer is formed on a contact surface where the first conductive lines and the second conductive lines cross, And conductive paths are selectively formed only in the resistance-variable layers at points where the nodes are formed, and the conductivity of the conductive paths may correspond to an element value of the kernel matrix.

In the embodiments of the present invention, the resistance-change memory may change resistance by movement of copper, silver or oxygen ions, and the first conductive line and the second conductive line are connected to the upper electrode of the resistance- The lower electrode can be formed.

The present invention provides a kernel hardware device having an extended acceptance area, which can output a feature map without requiring a scanning operation of a receptive field for a low-resolution image or minimizing a scanning operation for a high-resolution image have. Therefore, the time required for the scanning operation, the power consumption due to the external circuit, and the area of the external circuit can be reduced.

It also provides a parallel kernel hardware system in which the area of the crosspoint array structure is drastically reduced by applying the characteristics of a composite neural network requiring only localized connections.

Dimensional kernel hardware system in which the second conductive lines electrically connected to the output terminals are stacked in parallel with the substrate and the first conductive lines electrically connected to the input terminals are arranged perpendicularly to the substrate, It is advantageous for integration. It also allows an intuitive understanding of the kernel hardware system when the first conductive lines are arranged in a matrix.

The technical effects of the present invention are not limited to those mentioned above, and other technical effects not mentioned can be clearly understood by those skilled in the art from the following description.

1 is a schematic diagram for explaining the operation of a combined-product-neural network system;
2 is a schematic diagram showing a connection of an input data matrix, an output data matrix and a kernel matrix of a composite-object-based neural network system.
3 is a circuit diagram showing a kernel hardware system according to an embodiment of the present invention.
4 is a circuit diagram showing an orthogonal kernel hardware system in which a first straight line in which a plurality of input stages are arranged and a second straight line in which a plurality of output stages are arranged are orthogonal to each other according to an embodiment of the present invention.
FIG. 5 is a circuit diagram illustrating a parallel kernel hardware system in which a first straight line in which a plurality of input stages are arranged and a second straight line in which a plurality of output stages are arranged are parallel to each other according to an embodiment of the present invention.
6 is a comparative diagram for comparing areas of an orthogonal kernel hardware system and a parallel kernel hardware system.
7 is a cross-sectional view illustrating a three-dimensional kernel hardware system according to an embodiment of the present invention.
FIG. 8 is a top view illustrating a three-dimensional kernel hardware system according to an embodiment of the present invention.
FIG. 9 is a schematic diagram illustrating operations of a three-dimensional kernel hardware system according to an embodiment of the present invention.

While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. Rather, the intention is not to limit the invention to the particular forms disclosed, but rather, the invention includes all modifications, equivalents and substitutions that are consistent with the spirit of the invention as defined by the claims.

It will be appreciated that when an element such as a layer, region or substrate is referred to as being present on another element "on," it may be directly on the other element or there may be an intermediate element in between .

Although the terms first, second, etc. may be used to describe various elements, components, regions, layers and / or regions, such elements, components, regions, layers and / And should not be limited by these terms.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings. In the following description, the same reference numerals are used for the same constituent elements in the drawings and redundant explanations for the same constituent elements are omitted.

1 is a schematic diagram for explaining the operation of a combined-product-neural network system;

The composite neural network is constructed such that all the elements X1, X2, X3, ... constituting the input data 110 are not all connected to the elements Y1, Y2, Y3, ... constituting the output data -connected), each of the elements of the output data operates under the assumption that it is influenced only by a specific limited area among all the elements constituting the input data.

1, when the input data 110 is a 3 × 3 matrix and the output data 130 is a 2 × 2 matrix, one element Y1 of the output data matrix 130 is input data It can be seen that only the elements in the receptive field of the matrix are affected. The elements (X1, X2, X4, X5) in the first reception area of the input data are assigned weight values (K1, K2, K3, K4) for extracting features, Is an element Y1 of the output data. In this case, the weight has different values depending on the characteristics of the image to be extracted, and a matrix having weight values as elements is defined as a kernel matrix (Kernel Matrix).

In order to scan the input data 110, the acceptance area moves sequentially, and even if the acceptance area moves, the same kernel matrix is applied to derive an element of the output data. That is, the element Y2 of the output data is a sum of the inner product values of the elements K1, K2, K3, and K4 of the kernel matrix in the elements (X2, X3, X5, and X6) do.

2 is a schematic diagram showing a connection of an input data matrix, an output data matrix and a kernel matrix of a composite-object-based neural network system.

Referring to FIG. 2, when the elements K1, K2, K3, and K4 of the kernel matrix are colored and represented by lines, when the receiving area moves and the entire input data is scanned, the elements X1 , X2, X3 ...) and the elements (Y1, Y2, Y3, ...) of the output data matrix.

Y1 is connected to X1, X2, X4 and X5 via K1, K2, K3 and K4, and Y2 is connected to X2, X3, X5 and X6 through K1, K2, K3 and K4, respectively. That is, one output data element is connected to the input data element group, and the arrangement of weights assigned to the connection line is the same.

By implementing such a whole connection relationship in hardware, it is possible to minimize the movement of moving the receiving area and scanning the input data. That is, when the input image is a low-resolution image, output data can be derived without an operation of inputting and scanning the entire input data at a time. Alternatively, even when the input image is a high-resolution image, the input data to which the kernel hardware system of the present invention can be input is used as an extended receptive field, thereby minimizing the scanning operation.

Embodiment 1: An orthogonal kernel hardware system

3 is a circuit diagram showing a kernel hardware system according to an embodiment of the present invention.

Referring to FIG. 3, the kernel hardware system includes a plurality of input terminals 10 corresponding to the elements of the input data, a plurality of output terminals 30 corresponding to the elements of the output data, First conductive lines 40 electrically connected to the input terminals 10 and disposed in parallel to each other, second conductive lines 40 electrically connected to the output terminals 30 and intersecting the first conductive lines 40, And conductive lines 50.

The input data is divided into acceptance areas having a size of nxn, and the inputs corresponding to the elements constituting one acceptance area are defined as one input group. For example, the first input terminal group 11 corresponding to the elements constituting the first accommodation area includes X1, X2, X4, X5 input terminals.

The first conductive lines 41, 42, 44 and 45 electrically connected to the first input terminal group 11 are electrically connected to the second conductive line 51 through the first node group 21, . The electrical signals applied to the first input stage group 11 are weighted by the first node group 21 and are summed through the second-1 conductive line 51 and output to the Y1 output stage 31. Similarly, the first conductive lines 42, 43, 45, 46 electrically connected to the second input stage group 11 are electrically connected through the second-second conductive line 52 and the second node group 22 .

That is, the input group corresponding to the acceptance regions is electrically connected to the output node corresponding to the output data element affected by the acceptance region through the node group. On the other hand, these localized unaffected inputs and outputs are electrically isolated.

Each node constituting the node group 20 may be a resistance change memory. For example, the first conductive lines 40 and the second conductive lines 50 intersect each other to form an upper electrode and a lower electrode of the resistance change memory, and at the intersection of the conductive lines, A resistance variable layer can be formed between the first electrode and the second electrode.

Each node group 20 has a resistance value corresponding to an element value constituting the kernel matrix. That is, the weight value of the kernel matrix applied to extract features from the input data is converted into resistance or conductivity and stored in the resistance change memory. Since the same kernel matrix is applied to each receiving area, similarly each node group 20 applied to each input group has the same resistance arrangement. That is, the 1-1 node constituting the first node group 21 and the 2-1 node constituting the second node group 22 have the same resistance value, and the 1-2 node and the 2-2 node have the same resistance value Respectively.

The resistance change memory may change resistance or conductivity by movement of copper, silver or oxygen ions. That is, in the resistance change memory, ions are formed by a voltage difference between the first conductive line 40 and the second conductive line 50 corresponding to the upper electrode and the lower electrode, thereby changing the resistance or the conductivity value Can be stored.

A whole area of a low-resolution input image, or a part of a high-resolution input image, is input at a time in the form of matrix data through an input end. Each output stage is influenced only by the value input to the input stage group electrically connected through the node group 20. The electric signal applied to each input terminal by the current or voltage is converted by the conductivity or resistance value of the node, and the converted electric signals are added along the second conductive line and output through the output terminal.

4 is a circuit diagram showing an orthogonal kernel hardware system in which a first straight line in which a plurality of input stages are arranged and a second straight line in which a plurality of output stages are arranged are orthogonal to each other according to an embodiment of the present invention.

Referring to FIG. 4, it can be seen that the input data matrix has a size of 6 × 6, the output data matrix has a size of 4 × 4, and the receiving area has a size of 3 × 3. The number of input terminals, the number of output terminals, and the number of nodes constituting the node group can be changed according to the size of the input data, the size of the output data, and the size of the reception area.

A general crosspoint array in which a first straight line 71 in which input ends 10 are disposed and a second straight line 73 in which output ends 30 are disposed are orthogonal to each other, It can be confirmed that there are unnecessary intersection points. That is, the 1-1 conductive line 41 electrically connected to X1 has an intersection with the 16 second conductive lines 50, but the point where the node is formed is the 1-1 conductive line 41, And the second-first conductive line 51 intersect with each other. The remaining intersections are unnecessary areas that are not electrically connected.

Therefore, a structure for reducing power loss due to unnecessary wiring and minimizing a device to enable high integration is proposed.

Example 2: Parallel Kernel Hardware System

FIG. 5 is a circuit diagram illustrating a parallel kernel hardware system in which a first straight line in which a plurality of input stages are arranged and a second straight line in which a plurality of output stages are arranged are parallel to each other according to an embodiment of the present invention.

Referring to FIG. 5, the kernel hardware system according to the second embodiment includes a plurality of input terminals 10 disposed along a first straight line spaced from each other by a predetermined distance, a plurality of input terminals 10 spaced apart from each other by a predetermined distance, A plurality of output terminals 30 disposed along a straight line, first conductive lines 40 electrically connected to the input terminals 10 and arranged in parallel to each other, a plurality of first conductive lines 40 electrically connected to the output terminals 30, And second conductive lines (50) intersecting the first conductive lines (40).

The first conductive lines 40 and the second conductive lines 50 are orthogonal to each other and the conductive lines 40 and 50 extend only from the input or output end to the region where the nodes are formed.

For example, the first-first conductive line 41 connected to X1 is located at the intersection with the second-first conductive line 51 at the last node. Thus, the 1-1 conductive line 41 extends from the input terminal X1 to the intersection with the 2-1 conductive line 51. Likewise, the first-second conductive line 42 connected to X2 is located at a node that is the farthest from the input terminal X2 at a point intersecting the second-second conductive line 52. [ Thus, the 1-2 conductive line 42 extends from the input X2 to the intersection with the 2-2 conductive line 52. The same applies to the case of the second conductive lines 50. In the case of the second-first conductive line 51, the node furthest from the output end Y1 is located at the intersection with the 1-15th conductive line. Thus, the second-first conductive line 51 extends from the output terminal Y1 to the intersection with the 1-15th conductive line.

6 is a comparative diagram for comparing areas of an orthogonal kernel hardware system and a parallel kernel hardware system.

Referring to FIG. 6, in the case of the orthogonal kernel hardware system (a), the conductive lines extend to occupy a large area to unnecessary intersections where there is no electrical connection, while in the case of the parallel kernel hardware system (b) It can be confirmed that only the area formed by the node is occupied by removing most intersections.

Therefore, when the parallel kernel hardware system (b) according to the second embodiment is used, high integration of the system is possible and power consumption due to unnecessary conductive lines can be reduced. Also, since the input line and the output line are arranged in parallel, connection between the layers can be easily realized.

Embodiment 3: Three-dimensional kernel hardware system

7 is a cross-sectional view illustrating a three-dimensional kernel hardware system according to an embodiment of the present invention.

Referring to FIG. 7, a kernel hardware system according to the third embodiment includes a substrate 500, a plurality of input terminals 10 corresponding to elements of input data composed of a matrix, elements of output data composed of a matrix A plurality of output terminals 30 corresponding respectively to the first conductive lines 540 arranged on the substrate 500 and spaced apart from each other perpendicularly to the substrate 500. And second conductive lines 550 stacked on the substrate 500 in parallel with the substrate 500.

A resistance variable layer 570 is formed between the first conductive lines 540 and the second conductive lines 550 along the periphery of the first conductive lines 540.

The conductive paths are selectively formed only in the resistance variable layer at the point 560 where nodes are formed in the resistance variable layer 570 located at the intersection of the first conductive lines 540 and the second conductive lines 550 , And the other portions can maintain an insulated state or a high resistance state.

Insulating layers 580 may be formed between the second conductive lines 550 to insulate the second conductive lines 550 and to maintain the structure. That is, the second conductive lines 550 and the insulating layers 580 may have alternately stacked structures.

When the three-dimensional kernel hardware system according to the third embodiment is applied, it is possible to minimize the area required for the device and achieve high integration.

FIG. 8 is a top view illustrating a three-dimensional kernel hardware system according to an embodiment of the present invention.

Referring to FIG. 8, as shown in FIG. 7, input terminals 10 may be arranged in a line along a straight line or may be arranged in a matrix like input data. The input terminals 10 are not limited to those shown in this embodiment, and may have various arrangements in some cases.

FIG. 9 is a schematic diagram illustrating operations of a three-dimensional kernel hardware system according to an embodiment of the present invention.

Referring to FIG. 9 (a), the input terminals are arranged in a matrix as shown in FIG. 8 (b), and the first conductive lines 40 connected to the input ends also form a matrix. The cross section of the first conductive lines 40 and the second conductive line 50 is shown in a plan view in FIG. 9 (b).

In one embodiment, each of the conductive lines 50 has intersections arranged in a 3x3 matrix. At this time, the first node group 21 is formed at the intersection of (1,1), (1,2), (2,1), (2,2) among the intersections of the second- do. Similarly, a second node group 22 is formed at an intersection of (1,2), (1,3), (2,2), (2,3) among the intersections of the second-second conductive line 51 .

The input data is applied as an electrical signal through the first conductive lines 40 and at the same time to the second conductive lines 50 through the node groups 20. Each node group 20 has the same resistance arrangement. The applied input data is converted via the node groups 20 and the summed value through the second conductive lines 50 is output through the output stage.

The kernel hardware system according to the present invention can input the extended acceptance area or the entire image larger than the acceptance area, thereby minimizing the scanning operation. Also, the area occupied by the kernel hardware system is minimized, which enables high integration of the system.

110: input data matrix 120: kernel matrix
130: output data matrix
10: input stages 11: first input stage group
20: Node groups 21: First node group
30: output terminals 40: first conductive lines
50: second conductive lines 71: first straight line
73: 2nd straight line
500: substrate 540: first conductive lines
550: second conductive lines 560: nodes
570: resistance variable layer 580: insulating layers

Claims (8)

A plurality of input terminals respectively corresponding to elements of input data composed of a matrix;
A plurality of output terminals respectively corresponding to elements of output data composed of a matrix;
First conductive lines electrically connected to the input terminals and disposed parallel to each other;
And second conductive lines electrically connected to the output terminals and intersecting the first conductive lines,
Wherein the input data is divided into acceptance regions having a size of nxn so that input terminals corresponding to elements of one acceptance region are defined as one input terminal group,
The first conductive lines electrically connected to the input terminal group are electrically connected to one of the second conductive lines through the node group,
Wherein each node of the group of nodes is a resistive memory. The method according to claim 1,
Each node constituting one node group has a resistance value corresponding to an element value constituting the kernel matrix,
Wherein all of said node groups have the same resistance arrangement. The method according to claim 1,
Wherein the plurality of input terminals are disposed on a first straight line spaced apart from each other by a predetermined distance,
The plurality of output terminals are disposed on a second straight line spaced apart from each other by a predetermined distance and parallel to the first straight line,
Each of the first conductive lines extending from the input end to an area where the nodes are formed,
Each of the second conductive lines extending from the output end to an area where the nodes are formed,
Wherein the first conductive line and the second conductive line are orthogonal to each other. The method of claim 3,
Resistance change layers are formed between the first conductive lines and the second conductive lines at points where the first conductive lines and the second conductive lines cross vertically,
A conductive path is selectively formed only in the resistance variable layer at a point where the node is formed,
Wherein the conductivity of the conductive path corresponds to an element value of the kernel matrix. The method according to claim 1,
The first conductive lines being spaced apart from each other and arranged on the substrate perpendicular to the substrate,
The second conductive lines are spaced apart from each other and stacked on the substrate in parallel,
Wherein the first conductive lines and the second conductive lines are orthogonal to each other. 6. The method of claim 5,
The second conductive lines having an area,
The first conductive lines passing through the second conductive lines are arranged in a matrix,
Resistance-variable layers are formed on the contact surfaces where the first conductive lines and the second conductive lines intersect,
Conductive paths are selectively formed only in the resistance-variable layers at points where the nodes are formed,
Wherein the conductivity of the conductive path corresponds to an element value of the kernel matrix. The method according to claim 1,
Wherein the resistance change memory performs a resistance change by movement of copper, silver, or oxygen ions. The method according to claim 1,
Wherein the first conductive line and the second conductive line form an upper electrode and a lower electrode of the resistance-change memory, respectively.

Images