TWI880680B - Device for executing machine learning model inference - Google Patents

Abstract

A device for executing machine learning inference is provided. The device includes a computer and a trusted processing circuit. The computer receives a random seed generated by an electronic device. The trusted processing circuit includes a secure circuit and a processing element array. The secure circuit has a first public key and a first private key that are embedded in the secure circuit. The secure circuit generates a proof according to the random key, the first public key and the first private key. The computer outputs the proof and the first public key to the electronic device. The electronic device determines whether to output an encrypted machine learning model to the computer. The secure circuit decrypts the encrypted machine learning model to generate a machine learning model. The processing element array executes confidential computing of an inference of the machine learning model.

Description

Device for performing machine learning model inference

This disclosure relates to a device for executing a machine learning model, and in particular to a device for executing a machine learning model using identity authentication technology.

As the training cost of machine learning models gradually increases, the confidentiality protection of machine learning models has become an important issue in the sales and deployment of machine learning products. When a model provider sells or provides a machine learning model to a model user, he must only provide the user with permission to use the model, ensuring that others cannot easily obtain the confidential information of the model, including the structure and weight of the model. In addition, the professional data used by enterprises in executing machine learning model training or inference is also quite confidential, and the risk of confidential data leakage must be avoided as the model is used. This type of technology that provides models to users while protecting the confidentiality of the models is called secure computing or confidential computing.

According to some embodiments of the present disclosure, a device for performing machine learning model inference is provided. The device includes a host and a trusted processing circuit. The host receives a random seed generated by an electronic device. The trusted processing circuit includes a privacy circuit. The privacy circuit has an embedded first public key and a first private key, and the privacy circuit generates a certificate based on the random seed, the first public key, and the first private key. The host outputs the certificate and the first public key to the electronic device for the electronic device to determine whether to output the encrypted machine learning model to the host. The privacy circuit decrypts the encrypted machine learning model to generate a machine learning model. The processing element array securely calculates the inference operation of the machine learning model.

In some embodiments, the first public key is registered in a database so that the electronic device can query whether the trusted processing circuit is legitimate hardware based on the received first public key.

In some embodiments, the host includes a memory and a processor. The processor writes the encrypted machine learning model into the memory. The trusted processing circuit further includes a blind memory controller and a bus. The blind memory controller is coupled to the memory and reads the encrypted machine learning model from the memory. The bus transmits the encrypted machine learning model to the privacy circuit.

In some embodiments, the bus self-privacy circuit transmits the machine learning model to the obscure memory controller, and the obscure memory controller writes the machine learning model into the memory according to an access method different from that of the processor.

In some embodiments, the trusted processing circuit further includes a controller. The controller requires the oblivious memory controller to read the first layer of the machine learning model in the inference operation. The processing element array generates a first feature based on the input data and the first layer. The controller determines whether to store the first feature in the memory or the buffer memory of the trusted processing circuit according to the size of the first feature.

In some embodiments, the obscure memory controller writes the prediction data generated by the inference operation to the memory in the same manner as the processor accesses it.

According to some embodiments of the present disclosure, a device for performing machine learning model inference is provided, comprising a trusted processing circuit and a host. The trusted processing circuit comprises a privacy circuit. The privacy circuit has an embedded first public key and a first private key. The host outputs the first public key to an electronic device for the electronic device to check whether the first public key is registered in a database to determine whether to transmit an encrypted machine learning model to the host. The privacy circuit decrypts the encrypted machine learning model according to the first private key to generate a machine learning model. The trusted processing circuit securely calculates the inference operation of the machine learning model.

In some embodiments, the privacy circuit decrypts the encrypted machine learning model to confirm the usage restriction status. When the usage restriction status includes the number of uses, the trusted processing circuit prohibits the execution of the inference operation when the number of times the inference operation is performed is greater than the number of uses. When the usage restriction status includes the expiration date, the trusted processing circuit prohibits the execution of the inference operation when the current time is earlier than the expiration date.

In some embodiments, the privacy circuit decrypts the encrypted machine learning model to confirm the usage restriction status. When the usage restriction status includes a restricted signature status. The host blinds the input data to generate blinded input data, and transmits the blinded input data to the electronic device to perform a blind signature operation on the input data. The host receives the second public key of the electronic device and the blind signature generated by the electronic device based on the blinded input data, and unblinds the blind signature to generate a signature. The trusted processing circuit determines whether to perform an inference operation on the input data based on the second public key and the signature.

In some embodiments, the host includes a memory and a processor. The processor writes the encrypted machine learning model into the memory according to a first addressing method. The trusted processing circuit further includes a memory controller. The memory controller writes the machine learning model into the memory according to a second addressing method different from the first addressing method to prevent the processor from reading the machine learning model.

10: System

100: Electronic devices

110: Trusted processing circuit

111: Controller

112: Bus

113: Controller

114: Buffer memory

115: Privacy Circuit

116: Processing component array

120: Host

121: Processor

122: Memory

200: Electronic devices

201:Processor

202: Memory

300: Electronic devices

301:Processor

302: Memory

303: Database

20: Methods

S10-S14: Steps

S20-S24: Steps

S30-S32: Steps

S40: Step

S50-S56: Steps

S60: Step

In order to make the above and other purposes, features, advantages and embodiments of the present invention more clearly understandable, the attached drawings are described as follows: FIG. 1 is a schematic diagram of a system 10 according to some embodiments of the present disclosure; FIG. 2 is a flow chart of a method 20 for operating the system 10 shown in FIG. 1 according to some embodiments of the present disclosure; FIG. 3 is a flow chart of some steps of the method 20 shown in FIG. 2 according to some embodiments of the present disclosure; and FIG. 4 is a flow chart of some steps of the method 20 shown in FIG. 2 according to some embodiments of the present disclosure.

Various embodiments of the present disclosure will be described below with reference to the attached drawings. In the drawings, the same reference numerals represent the same or similar elements or method flows. On the other hand, well-known elements and steps are not described in the embodiments to avoid unnecessary limitations on the present invention.

Unless otherwise specified, the terms used in this disclosure generally have their ordinary meanings in this field. However, those with ordinary knowledge in the relevant technical field should understand that the same element or process can be represented by different terms.

The terms "first", "second", etc. used herein do not specifically refer to order or sequence, nor are they used to limit the present invention. They are only used to distinguish between elements or operations described with the same technical terms. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element. Such changes do not deviate from the scope of the disclosed embodiments.

Please refer to FIG. 1. FIG. 1 is a schematic diagram of a system 10 according to some embodiments of the present disclosure. The system 10 is used to perform secure computing of a machine learning model to ensure that the model confidentiality of the model provider and the data confidentiality of the model user will not be leaked during the execution of the machine learning model.

For illustration, the system 10 includes an electronic device 100 of a model user, an electronic device 200 of a model provider, and an electronic device 300 of a hardware provider. The electronic device 200 is used to store a machine learning model and provide a protected or encrypted machine learning model to the electronic device 100. The electronic device 100 is used to store test data and perform inference of the machine learning model on the test data. The electronic device 300 is used to check whether the hardware of the electronic device 100 has a qualified identity.

As shown in FIG. 1, the electronic device 100 includes a trusted processing circuit 110 and a host 120. The trusted processing circuit 110 is a processing circuit that supports trusted execution, such as a graphics processing unit circuit that supports trusted execution. In some embodiments, the trusted processing circuit 100 is a trusted neural processing unit (TNPU).

According to some embodiments, the trusted neural network processing unit is a secure neural network accelerator that ensures the confidentiality of user data and model parameters in an untrusted environment through hardware. Since the trusted neural network processing unit can be customized according to the neural network, the trusted neural network processing unit has a higher efficiency in secure inference of the neural network than some methods.

In some embodiments, the trusted processing circuit 110 includes a controller 111, a bus 112, a controller 113, a buffer 114, a privacy circuit 115, and a processing element array 116. In some embodiments, the bus is a system on a chip (SoC) bus. The controller 113 is an oblivious random access memory (ORAM) controller. The buffer 114 is an on-chip buffer. The privacy circuit 115 includes a privacy module circuit that supports security-related functions.

In some embodiments, the host 120 and the electronic devices 200-300 are computer hosts, such as laptops, desktop computers, or mainframe computers. The host 120, the electronic devices 200 and 300 include processors 121, 201, 301, and memories 122, 202, 302, respectively. The electronic device 300 further includes a database 303.

According to some embodiments, processors 121, 201, and 301 include a central processing unit (CPU), or other programmable general-purpose or special-purpose microcontrol unit (MCU), microprocessor, digital signal processor (DSP), programmable controller, application specific integrated circuit (ASIC), graphics processing unit (GPU), arithmetic logic unit (ALU), complex programmable logic device (CPLD), field programmable gate array (FPGA), or other similar components or combinations of the above components.

The memories 122, 202, and 302 respectively include static random access memory (SRAM), dynamic random access memory (DRAM), other suitable memories, or a combination of the above.

In the embodiment shown in FIG. 1, the trusted processing circuit 110 is coupled to the host 120. The host 120 is coupled to the electronic device 200. The electronic device 200 is coupled to the electronic device 300. It should be understood that in the embodiment, the description involving "coupling" may generally refer to one component being indirectly electrically coupled to another component through other components, or one component being directly electrically coupled to another component without passing through other components.

For example, in some embodiments, the trusted processing circuit 110 is directly coupled to an input/output (I/O) interface of the host 120. The host 120 is connected to the electronic device 200 via the Internet. The electronic device 200 is connected to the electronic device 300 via the Internet.

In some embodiments, bus 112 couples controllers 111 and 113, buffer memory 114, privacy circuit 115, and processing element array 116. Controller 111 couples processor 121. Controller 113 couples memory 122. Processor 121 couples memory 122. Processor 201 couples memory 202. Processor 301 couples memory 302 and database 303.

According to some embodiments, when the trusted processing circuit 110 leaves the factory, the identity (ID) and public key pkt of the trusted processing circuit 110 are registered in the electronic device 300. In some embodiments, the database 303 is used to store the identities and public keys of all trusted processing circuits manufactured by the hardware provider, including the ID and public key pkt of the trusted processing circuit 110. In some embodiments, the database 303 is an unalterable database, such as a blockchain database or a database secured by any suitable tool.

The processor 301 is used to perform query operations of the database 303 and secure communication between the electronic devices 300 and 200. Secure communication ensures that the confidential part of the data transmitted between the two devices will not be leaked by means of encryption, for example. The memory 302 is used to perform data temporary storage operations of the electronic device 300.

The processor 201 is used to perform secure communication with the trusted processing circuit 110, the host 120 and the electronic device 300. The processor 201 is further used to encrypt the machine learning model stored in the electronic device 200. The processor 201 is further used to perform an identification scheme for the electronic device 100. The processor 201 is further used to perform a blind signature operation on the data from the electronic device 100. The memory 202 is used to perform data temporary storage operations on the electronic device 200.

The processor 121 is used to perform secure communication with the electronic device 200 and the trusted processing circuit 110. The processor 201 is further used to transmit data to the trusted processing circuit 110 to perform secure computation of the encrypted machine learning model. The processor 201 is further used to request the electronic device 200 to perform a blind signature.

In some embodiments, the controller 111 is used to receive instructions from the processor 121. In some embodiments, the controller 111 and the privacy circuit 115 are used to perform secure communications with the electronic device 200 and the host 120, respectively.

The bus 112 is used to transmit data and signals between the controller 111, the controller 113, the buffer memory 114, the privacy circuit 115 and the processing element array 116.

The controller 113 is used to read and write to the memory 122. The controller 113 is used to protect the data access between the trusted processing circuit 110 and the memory 122, ensuring that the host 120 cannot obtain the confidentiality of the data written by the controller 113 to the memory 122. Specifically, in some embodiments, the controller 113 uses an access pattern different from that of the processor 121 to protect the confidentiality of the data written by the controller 113 to the memory 122.

The buffer memory 114 is used to perform temporary storage operations when executing internal operations of the trusted processing circuit 110.

In some embodiments, the trusted processing circuit 110 is used to store its own ID, public key pkt, and private key skt. Devices or circuits other than the trusted processing circuit 110 (such as the host 120) cannot access the private key skt, so the model user cannot obtain the private key skt. In some embodiments, one or more of the ID, public key pkt, and private key skt of the trusted processing circuit 110 is embedded in the privacy circuit 115. For example, in some embodiments, one or more of the ID, public key pkt, and private key skt of the trusted processing circuit 110 is the password of the physical unclonable function (PUF) of the privacy circuit 115. For example, the privacy circuit 115 includes a one-time programmable (OTP) memory storing one or more of the ID, public key pkt, and private key skt of the trusted processing circuit 110.

The processing element array 116 is used to perform inference acceleration operations of the machine learning model.

The configuration of FIG. 1 is provided for illustrative purposes. Various embodiments based on FIG. 1 are within the contemplated scope of the present disclosure. For example, the coupling shown in FIG. 1 can be replaced by signal connection methods such as wireless transmission, optical transmission, etc. For example, electronic device 100 is connected to electronic device 200 via wireless communication. Electronic device 200 is connected to electronic device 300 via wireless communication.

Please refer to Figures 2 to 4. Figure 2 is a flow chart of a method 20 for operating the system 10 shown in Figure 1 according to some embodiments of the present disclosure. Figures 3 and 4 are flow charts of some steps of the method 20 shown in Figure 2 according to some embodiments of the present disclosure.

As shown in FIG. 2, method 20 includes steps S10-S60. In some embodiments, as shown in FIG. 3, step S10 includes steps S11-S14 and step S20 includes steps S21-S24. In some embodiments, as shown in FIG. 4, step S30 includes steps S31-S32 and step S50 includes steps S51-S56.

In some embodiments, method 20 operates system 10 to perform secure machine learning model inference, and ensures the confidentiality of data transmission and machine learning model inference through identity authentication methods, encrypted machine learning models, blind signature input data, secure communication, and secure execution.

Please refer to Figures 1 to 3 together. Steps S10-S20 execute the identity verification method of the trusted processing circuit 110. In step S10, the system 10 generates a proof of the trusted processing circuit 110. In step S20, the system 10 verifies the proof of the trusted processing circuit 110.

As shown in FIG. 3, in step S11, the processor 201 executes the random seed generation operation RandomSeed() to generate a random seed R.

In step S12, the processor 201 performs secure communication with the trusted processing circuit 110 to output the random seed R to the trusted processing circuit 110. In some embodiments, the processor 201 outputs the random seed R to the host 120, and the host 120 outputs the random seed R to the trusted processing circuit 110.

In step S13, the electronic device 100 generates a certificate P proving that it has a legitimate trusted processing circuit 110. In some embodiments, the trusted processing circuit 110 performs a commit operation Commit(R, pkt, skt) of the identity verification method according to the random seed R, the public key pkt, and the private key skt to generate the certificate P. In some embodiments, the controller 111 receives an instruction from the processor 121 to request the privacy circuit 115 to perform the commit operation.

In step S14, the trusted processing circuit 110 performs secure communication to output the certificate P, public key pkt and ID to the host 120. The host 120 outputs the certificate P, public key pkt and ID to the electronic device 200.

In step S21, the processor 201 performs a verification operation Verify(pkt, P, R) of the identity verification method according to the public key pkt, the proof P and the random seed R to generate a verification result V. The processor 201 determines whether the trusted processing circuit 110 has a private key skt matching the public key pkt according to the verification result V. For example, when the verification result V has a first value, such as True, the processor 201 determines that the trusted processing circuit 110 has a private key skt corresponding to the public key pkt. On the contrary, when the verification result V has a second value different from the first value, such as False, the processor 201 determines that the trusted processing circuit 110 does not have a private key skt corresponding to the public key pkt.

In step S22, the processor 201 performs secure communication to output the public key pkt and ID to the electronic device 300. In some embodiments, when the processor 201 determines that the trusted processing circuit 110 has a private key skt corresponding to the public key pkt, the processor 201 outputs the public key pkt and ID to the electronic device 300. When the processor 201 determines that the trusted processing circuit 110 does not have a private key skt corresponding to the public key pkt, the processor 201 does not output the public key pkt and ID to the electronic device 300.

In step S23, the processor 301 performs a check operation Check(pkt, ID) based on the public key pkt and ID to generate a check result C. In the check operation Check(pkt, ID), the processor 301 queries whether the public key pkt and ID from the processor 201 are registered (i.e., whether they are recorded in the same registration data in the database 303). When the public key pkt and ID are registered, it means that the trusted processing circuit with the public key pkt and ID has been shipped. When the processor 301 determines that the public key pkt and ID are registered, the processor 301 generates a check result C with a first value (e.g., True). When the processor 301 determines that the public key pkt and ID are not registered, the processor 301 generates a check result C having a second value (e.g., False) different from the first value.

In step S24, the processor 301 executes secure communication to output the check result C to the electronic device 200. In some embodiments, the processor 201 determines whether the public key pkt and ID are registered based on the check result C. When the check result C has a first value (e.g., True), the processor 201 determines that the public key pkt and ID are registered. On the contrary, when the check result C has a second value different from the first value (e.g., False), the processor 201 determines that the public key pkt and ID are not registered.

In some embodiments, the processor 201 determines whether the electronic device 100 has passed the verification (i.e., whether it has a legitimate trusted processing circuit 110) based on the verification result V and the check result C. When the verification result V and the check result C have a first value (e.g., True), the processor 201 determines that the electronic device 100 has passed the verification. When the verification result V and/or the check result C have a second value (e.g., False), the processor 201 determines that the electronic device 100 has passed the verification. In some embodiments, the electronic device 100 passing the verification means that only the trusted processing circuit 110 has the private key skt that can decrypt the public key pkt.

Please refer to Figures 1 to 4 together. In step S30, the system 10 encrypts the machine learning model. In some embodiments, when the electronic device 100 is judged to have passed the verification in step 20, the system 10 executes step S30. On the contrary, when the electronic device 100 is judged to have failed the verification in step 20, the system 10 does not execute step S30.

In step S31, the processor 201 performs an encryption operation Enc(pkt,M) according to the public key pkt to encrypt the machine learning model M stored in the electronic device 200 to generate an encrypted machine learning model M'.

In some embodiments, the model provider determines the usage restrictions of the machine learning model M. During the encryption process, the processor 201 stores a restriction state indicating the usage restrictions of the machine learning model M in the encrypted machine learning model M'.

In some embodiments, the model provider determines whether the input data (test data) needs to be signed to limit the input data that the model user can perform inference on. When the model provider determines that the input data (test data) needs to be signed, the processor 201 stores a restriction state including state s in the encrypted machine learning model M' in step S31, and the state s indicates that the input data (test data) needs to be signed.

In some embodiments, the model provider determines the number of times the machine learning model M is used to limit the number of times the model user can perform inference. When the model provider determines that the machine learning model M has a number of usage restrictions, the processor 201 stores a restriction state including state c in the encrypted machine learning model M' in step S31, and the state c indicates that the machine learning model M has a number of usage restrictions. In some embodiments, state c includes the number of times N that the machine learning model M is restricted from being used.

In some embodiments, the model provider determines the usage time of the machine learning model M to limit the period during which the model user can perform inference. When the model provider determines that the machine learning model M has a usage time limit, the processor 201 stores a restriction state including a state t in the encrypted machine learning model M' in step S31, and the state t indicates that the machine learning model M has a usage time limit. In some embodiments, the state t includes the usage time T of the machine learning model M. In some embodiments, the usage time T is the usage period of the machine learning model M.

In some embodiments, when the model provider determines that the machine learning model M has no usage restrictions, the processor 201 stores the restriction state including the state n in the encrypted machine learning model M' in step S31, and the state n indicates that the machine learning model M has no usage restrictions.

In step S32, the processor 201 performs secure communication with the trusted processing circuit 110 to output the encrypted machine learning model M' to the trusted processing circuit 110. In some embodiments, the processor 121 receives the encrypted machine learning model M' and outputs the encrypted machine learning model M' to the controller 111.

In step S40, the system 10 determines whether the input data (test data) of the machine learning model M needs to be signed. In some embodiments, the processor 121 executes step S50 based on the determination that the restricted state includes state s (i.e., the machine learning model M only accepts signed input data). On the contrary, when the processor 121 determines that the restricted state does not include state s, the system 10 does not execute step S50.

In step S50, the system 10 blindly signs the input data (test data) X stored in the electronic device 100 to generate signed test data that the machine learning model M is allowed to use. The input data X is the test data that the model user wants to use the machine learning model M to infer.

In step S51, the processor 121 performs a blinding operation Blinding(X) in a blind signature on the input data X to generate a blinded data X'.

In step S52, the processor 121 performs secure communication with the electronic device 200 to output the blinded data X' to the electronic device 200.

In step S53, the processor 201 executes the blind signature key generation operation GenSignKey to generate the private key sks and the public key pks. The system 10 performs blind signing based on the private key sks and the public key pks.

In step S54, the processor 201 performs the sign operation Sign(sks,X') in the blind signature on the blinded data X' according to the private key sks to generate a blinded signature S'.

In step S55, the processor 201 performs secure communication with the host 120 to output the blinded signature S' to the host 120.

In some embodiments, the processor 201 further performs secure communication with the trusted processing circuit 110 to output the public key pks to the trusted processing circuit 110.

In step S56, the processor 121 performs the unblinding operation UnBlinding(S') on the blinded signature S' to generate the signature S.

As shown in FIG. 4, when the system 10 determines in step S40 that the input data needs to be signed, the system 10 executes step S50 and then executes step S60. Conversely, when the system 10 determines in step S40 that the input data does not need to be signed, the system 10 directly executes step S60.

In step S60, the system 10 securely executes the inference of the machine learning model M. In some embodiments, the controller 103 is used to perform oblivious read and write operations (e.g., using a different addressing method from the processor 121) to achieve secure execution of the inference.

In some embodiments, the processor 121 stores the encrypted machine learning model M' in the memory 122 in step S60.

The controller 111 is used to request the privacy circuit 115 to decrypt the encrypted machine learning model M' into the machine learning model M. The controller 113 stores the machine learning model M in the memory 122. In some embodiments, the controller 113 writes the machine learning model M into the memory 122 invisibly.

In some embodiments, the processor 121 stores the input data X in the memory 122, and the input data X serves as the input data for the controller 111 to perform the inference operation of the machine learning model M.

The controller 111 decrypts the machine learning model M' to obtain the restriction state of the machine learning model M, and the controller 111 confirms the state included in the restriction state of the machine learning model M.

When the controller 111 confirms that the restricted state of the machine learning model M includes state s (i.e. the input data needs to be signed), the processor 121 stores the signature S and the public key pks in the memory 122.

Next, the controller 111 performs a verification operation Verify(pks,X,S) in the blind signature to determine whether the signature S is correct (determine whether the input data X matches the signature S) based on the public key pks and the input data X. In some embodiments, the controller 111 performs a verification operation Verify(pks,X,S) based on the public key pks, the input data X, and the signature S to generate a verification result. When the verification result has a first value (e.g., True), the controller 111 determines that the signature S is correct. Conversely, when the verification result has a second value different from the first value (e.g., False), the controller 111 determines that the signature S is incorrect.

When the controller 111 determines that the signature S is correct, the controller 111 then performs the model inference operation Model(M,X) of the machine learning model M on the input data X to generate the prediction data Y.

When the controller 111 confirms that the restricted state of the machine learning model M includes state c (i.e., there is a usage limit), the controller 111 stores a count value in the buffer memory 114. In some embodiments, the initial value of the count value is a value "0" or "1". The controller 111 adds a value "1" to the count value each time it executes the model inference operation Model(M,X). When the count value is greater than the restricted usage number N, the controller 111 no longer executes (prohibits) the model inference operation Model(M,X).

When the controller 111 confirms that the restricted state of the machine learning model M includes state t (i.e., there is a usage time limit), the controller 111 confirms whether the current time is within the restricted usage time T before each execution of the model inference operation Model(M,X). When the controller 111 confirms that the current time is earlier than or equal to the restricted usage time T, the controller 111 executes the model inference operation Model(M,X). When the controller 111 confirms that the current time is later than the restricted usage time T, the controller 111 no longer executes the model inference operation Model(M,X).

When the controller 111 confirms that the restricted state of the machine learning model M includes state n (i.e., there is no restriction), the controller 111 directly executes the model inference operation Model(M,X).

According to some embodiments, in the model inference operation Model(M,X), the controller 111 requests the controller 113 to read the first layer (e.g., including the weights of the first layer) of the machine learning model M from the memory 122. The processing element array 116 generates the first layer of features according to the input data X and the first layer of the machine learning model M. The controller 111 determines the size (occupied capacity) of the first layer of features and determines whether to store the first layer of features in the buffer memory 114 or the memory 122. When the size of the first layer of features is less than a threshold, the first layer of features is stored in the buffer memory 114. On the contrary, when the size of the first layer feature is greater than this threshold, the first layer feature is stored in the memory 122.

Next, the controller 111 requests the controller 113 to read the second layer of the machine learning model M from the memory 122, and the processing element array 116 generates the second layer features according to the first layer features of the machine learning model M and the second layer inference of the machine learning model M. Similar to the operation described in the previous paragraph, the controller 111 determines the size of the second layer features and stores the second layer features in the buffer memory 114 or the memory 122.

The controller 111 repeatedly performs operations similar to the above for the remaining layers of the machine learning model M to generate features and prediction data Y for each layer.

The controller 111 requests the controller 113 to store the prediction data Y in the memory 122 in an unencrypted form (e.g., without using a blind write operation). When the controller 111 completes the model inference operation Model(M,X), the controller 111 generates an end signal to the processor 121, indicating that the model inference operation Model(M,X) is terminated.

It should be understood that the steps of method 20 shown in Figures 2 to 4, except for those specifically described in order, can be adjusted in order according to actual needs, and can even be performed simultaneously or partially simultaneously. Additional operations can be provided before, during, and after the steps shown in Figures 2 to 4, and some of the operations described above can be replaced or eliminated to obtain additional embodiments of method 20.

In summary, the present disclosure provides a system and method for performing machine learning model inference. The system provided by the present disclosure encrypts the machine learning model of the model provider to protect the confidentiality of the model. In addition, the system performs an identity verification method for the trusted processing circuit of the model user to ensure that only this trusted processing circuit can decrypt the encrypted machine learning model. The system provided by the present disclosure further supports secure communication and secure computing to ensure that the confidentiality of the model will not be leaked.

The features of several embodiments are summarized above so that those skilled in the art can better understand the various aspects of an embodiment of the present invention. Those skilled in the art should understand that they can easily use an embodiment of the present invention as a basis for designing or modifying other processes and structures to achieve the same purpose and/or the same advantages of the embodiments introduced herein. Those skilled in the art should also recognize that these equivalent structures do not deviate from the spirit and scope of an embodiment of the present invention, and these equivalent structures can be variously modified, replaced and altered herein without departing from the spirit and scope of an embodiment of the present invention.

10: System

100: Electronic devices

110: Trusted processing circuit

111: Controller

112: Bus

113: Controller

114: Buffer memory

115: Privacy Circuit

116: Processing component array

120: Host

121: Processor

122: Memory

200: Electronic devices

201:Processor

202: Memory

300: Electronic devices

301:Processor

302: Memory

303: Database

Claims (10)

A device for performing machine learning model inference, comprising: a host for receiving a random seed generated by an electronic device; and a trusted processing circuit, comprising: a privacy circuit having an embedded first public key and a first private key, the privacy circuit for generating a certificate based on the random seed, the first public key and the first private key, wherein the host is further used to output the certificate and the first public key to the electronic device for the electronic device to determine whether to output an encrypted machine learning model to the host, wherein the privacy circuit is further used to decrypt the encrypted machine learning model to generate a machine learning model; and an array of processing elements for securely computing an inference operation of the machine learning model. The device as described in claim 1, wherein the first public key is registered in a database so that the electronic device can query whether the trusted processing circuit is legitimate hardware based on the received first public key. The device as described in claim 1, wherein the host comprises: a memory; and a processor for writing the encrypted machine learning model into the memory, wherein the trusted processing circuit further comprises: a blind memory controller coupled to the memory and for reading the encrypted machine learning model from the memory; and a bus for transmitting the encrypted machine learning model to the privacy circuit. The device as described in claim 3, wherein the bus is further used to transmit the machine learning model from the privacy circuit to the obscure memory controller, wherein the obscure memory controller is further used to write the machine learning model into the memory according to an access method different from that of the processor. The device as described in claim 4, wherein the trusted processing circuit further comprises: A controller for requesting the oblivious memory controller to read a first layer of the machine learning model in the inference operation, wherein the processing element array is further used to generate a first feature according to an input data and the first layer, wherein the controller is further used to determine whether to store the first feature in the memory or a buffer memory of the trusted processing circuit according to the size of the first feature. A device as described in claim 4, wherein the obscure memory controller is further used to write a prediction data generated by the inference operation into the memory according to an access method similar to that of the processor. A device for performing machine learning model inference, comprising: A trusted processing circuit, comprising: A privacy circuit, having a first public key and a first private key embedded therein; and A host, for outputting the first public key to an electronic device, so that the electronic device can check whether the first public key is registered in a database to determine whether to transmit an encrypted machine learning model to the host, wherein the privacy circuit is further used to decrypt the encrypted machine learning model according to the first private key to generate a machine learning model; and wherein the trusted processing circuit is used to securely calculate an inference operation of the machine learning model. The device as described in claim 7, wherein the privacy circuit is further used to decrypt the encrypted machine learning model to confirm a usage restriction state; wherein when the usage restriction state includes a usage count, the trusted processing circuit prohibits the execution of the inference operation when the number of times the inference operation is executed is greater than the usage count; and wherein when the usage restriction state includes a usage limit, the trusted processing circuit prohibits the execution of the inference operation when the current time is earlier than the usage limit. A device as described in claim 7, wherein the privacy circuit is further used to decrypt the encrypted machine learning model to confirm a usage restriction state; wherein when the usage restriction state includes a restricted signature state, the host is further used to blind an input data to generate a blinded input data, and transmit the blinded input data to the electronic device to perform a blind signature operation on the input data; wherein the host is further used to receive a second public key of the electronic device and a blinded signature generated by the electronic device based on the blinded input data, and unblind the blinded signature to generate a signature; and wherein the trusted processing circuit determines whether to perform the inference operation on the input data based on the second public key and the signature. A device as described in claim 7, wherein the host comprises: a memory; and a processor for writing the encrypted machine learning model into the memory according to a first addressing method, wherein the trusted processing circuit further comprises: a memory controller for writing the machine learning model into the memory according to a second addressing method different from the first addressing method to prevent the processor from reading the machine learning model.

Images