[go: up one dir, main page]

WO2025144460A1 - Codage randomisé collaboratif permettant d'entraîner des modèles appris par machine avec des données d'entraînement divisées verticalement - Google Patents

Codage randomisé collaboratif permettant d'entraîner des modèles appris par machine avec des données d'entraînement divisées verticalement Download PDF

Info

Publication number
WO2025144460A1
WO2025144460A1 PCT/US2024/035818 US2024035818W WO2025144460A1 WO 2025144460 A1 WO2025144460 A1 WO 2025144460A1 US 2024035818 W US2024035818 W US 2024035818W WO 2025144460 A1 WO2025144460 A1 WO 2025144460A1
Authority
WO
WIPO (PCT)
Prior art keywords
computing system
dataset
data
encoded
machine
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/US2024/035818
Other languages
English (en)
Inventor
Karn SETH
Mihaela Ion
Benjamin KREUTER
Mariana Petrova RAYKOVA
Marcel M.M. Yung
Sarvar Patel
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Google LLC
Original Assignee
Google LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Google LLC filed Critical Google LLC
Priority to EP24745567.8A priority Critical patent/EP4599421A1/fr
Publication of WO2025144460A1 publication Critical patent/WO2025144460A1/fr
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09C—CIPHERING OR DECIPHERING APPARATUS FOR CRYPTOGRAPHIC OR OTHER PURPOSES INVOLVING THE NEED FOR SECRECY
    • G09C1/00—Apparatus or methods whereby a given sequence of signs, e.g. an intelligible text, is transformed into an unintelligible sequence of signs by transposing the signs or groups of signs or by replacing them by others according to a predetermined system
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
    • H04L9/3236—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials using cryptographic hash functions
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L2209/00—Additional information or applications relating to cryptographic mechanisms or cryptographic arrangements for secret or secure communication H04L9/00
    • H04L2209/04—Masking or blinding
    • H04L2209/046—Masking or blinding of operations, operands or results of the operations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L2209/00—Additional information or applications relating to cryptographic mechanisms or cryptographic arrangements for secret or secure communication H04L9/00
    • H04L2209/46—Secure multiparty computation, e.g. millionaire problem
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L2209/00—Additional information or applications relating to cryptographic mechanisms or cryptographic arrangements for secret or secure communication H04L9/00
    • H04L2209/50—Oblivious transfer

Definitions

  • the present disclosure relates generally to secure multi-party computation. More particularly, example aspects of the present disclosure relate to allowing for multi-party computation of a joint function while preventing leaking or sharing of any private data.
  • Multi-party computation provides for computing on private datasets without sharing their input data with each other. Multi-party computation allows for collaboration and computation on private datasets without the parties sharing any of the private data with other parties.
  • the present disclosure provides for an example computer- implemented method.
  • the example computer-implemented method includes maintaining, by a first computing system, a machine-learned model; generating a first encoded dataset by sealing, by the first computing system using a shared key, a first private dataset into a fixed joint computation; storing, by the first computing system, the first encoded dataset; deleting, by the first computing system, the shared key; obtaining, by the first computing system, a second encoded dataset from a second computing system, wherein the second encoded dataset was generated by the second computing system using the shared key to seal a second private dataset into the fixed joint computation; generating, by the first computing system, an output by evaluating the fixed joint computation using the first encoded dataset and the second encoded dataset as input; and updating, by the first computing system, the machine- learned model based on the output.
  • the present disclosure provides for an example computer- implemented method.
  • the example computer-implemented method includes maintaining a machine-learned model comprising a plurality of weights; generating, by a first computing system, using input data comprising a batch of a first private dataset and a hash seed derived from a shared secret, a plurality of buckets of data comprising a bucket ID, a hashed identifier, and a hashed feature; generating, by the first computing system, a first set of random encoded data of the batch of the first private dataset by: for each bucket of the plurality of buckets: (i) generating a random vector; (ii) determining an error by performing backpropagation of the machine-learned model using input comprising the hashed feature and the random vector; (iii) computing a garbled circuit; and (iv) storing wire labels of the garbled circuit for the respective bucket; generating, by the first computing system, a first sum of errors by summing
  • the present disclosure provides for an example system for collaborative randomized encodings, including one or more processors and one or more memory devices storing instructions that are executable to cause the one or more processors to perform operations.
  • the one or more memory devices can include one or more transitory or non-transitory computer-readable media storing instructions that are executable to cause the one or more processors to perform operations.
  • the operations can include maintaining, by a first computing system, a machine- learned model; generating a first encoded dataset by sealing, by the first computing system using a shared key, a first private dataset into a fixed joint computation; storing, by the first computing system, the first encoded dataset; deleting, by the first computing system, the shared key; obtaining, by the first computing system, a second encoded dataset from a second computing system, wherein the second encoded dataset was generated by the second computing system using the shared key to seal a second private dataset into the fixed joint computation; generating, by the first computing system, an output by evaluating the fixed joint computation using the first encoded dataset and the second encoded dataset as input; and updating, by the first computing system, the machine-learned model based on the output.
  • the present disclosure provides for an example transitory or non-transitory computer readable medium embodied in a computer-readable storage device and storing instructions that, when executed by a processor, cause the processor to perform operations.
  • the operations include maintaining, by a first computing system, a machine-learned model; generating a first encoded dataset by sealing, by the first computing system using a shared key, a first private dataset into a fixed joint computation; storing, by the first computing system, the first encoded dataset; deleting, by the first computing system, the shared key; obtaining, by the first computing system, a second encoded dataset from a second computing system, wherein the second encoded dataset was generated by the second computing system using the shared key to seal a second private dataset into the fixed joint computation; generating, by the first computing system, an output by evaluating the fixed joint computation using the first encoded dataset and the second encoded dataset as input; and updating, by the first computing system, the
  • Figure 1 an example system for collaborative randomized encodings according to example aspects of the present disclosure
  • Figure 2 depicts a swim lane diagram of an example data flow to perform collaborative randomized encoding according to example aspects of the present disclosure
  • Figure 3 depicts a swim lane diagram of an example data flow to perform collaborative randomized encoding according to example aspects of the present disclosure
  • Figure 4 depicts a swim lane diagram of an example data flow to perform collaborative randomized encoding according to example aspects of the present disclosure
  • Figure 5 depicts a flowchart of example methods for performing collaborative randomized encodings according to example aspects of the present disclosure
  • Figure 6A and Figure 6B depict a flowchart of example methods for performing collaborative randomized encodings according to example aspects of the present disclosure.
  • Figure 7 depicts an example system to perform collaborative randomized encodings according to example aspects of the present disclosure.
  • collaborative randomized encodings can be used to perform joint computation across multiple parties with vertically split data.
  • the parties can use a shared secret, such as a cryptographic key, to seal their respective datasets into a fixed computation.
  • the shared secret can be subsequently deleted. After the secret is deleted, the sealed data is cryptographically random, and the parties cannot recover the data but can carry out the computation. After the secret is deleted, the parties can share the randomized data to perform the computation but cannot use the data for anything else or determine anything about the original dataset.
  • Standard secure multi-party computation provides for parties retaining their secret keys during computation. If these keys are leaked during computation, the data could be recovered based on the exchanged messages together from the secret keys of the parties.
  • the present disclosure provides for improved methods to delete the keys and secrets used to encrypt or randomize the data and fix the data in the computation before any data-derived values are exchanged between the parties. The parties can still perform the joint computation by combining the two (or more) sets of randomized data to complete the computation.
  • the present disclosure provides for a method for performing secure multi-party computation for any joint function using an application of garbled circuits that provides for protection in settings where both parties’ data is protected after key-deletion or where only one party’s data is protected after key -deletion.
  • Implementations of this technology can include training machine-learned models with vertically split training data, determining intersection of sums, or performing other joint computations. More specifically, the present disclosure can provide for an implementation for training machine-learned models with vertically split training data (e.g., vertical federated machine learning). For instance, gradients for machine-learned models can be computed over vertically split data where a first computing party holds identifiers and associated features, and a second computing party holds identifiers and associated training labels.
  • vertical federated machine learning e.g., vertical federated machine learning
  • the private datasets can be encrypted in such a way that a party holding the model can perform backpropagation with the first party’s encoded data and the second party’s encoded data and use the outputs to determine a gradient and adjust the weights of the machine-learned model accordingly.
  • the present disclosure allows for training of a model held by one of the parties without the two parties having to share any private data.
  • systems and methods according to example aspects of the present disclosure can enable, for example, training of machine-learned models while maintaining differential privacy for datasets including overlapping samples without requiring exponentially more utilization of computation resources.
  • the properties of a garbled circuit can be utilized to perform a value-or-default computation to allow for parallelization of training and processing.
  • the present systems and methods provide for improved data security by decreasing potential data leakage and providing for differentially private training using datasets where the features and labels of the training data are split among multiple parties. More particularly, the present method provides for improved data security by requiring a shared key to be destroyed before any encoded data is transferred between parties. This allows for the randomly encoded data to be shared without risk of private data leaking.
  • the methods described herein can be used for training a machine-learned model that predicts likely future diagnoses of illness based on features (e.g., risk factors). This can allow for multiple healthcare parties to utilize their data for training the model without sharing the original private data.
  • applications can include predicting likely interests or topics for a user based on features associated with a device identifier. Outside of performing vertical federated learning, the methods described herein can help perform aggregate computing operations to determine an intersection sum or to determine other aggregated computations associated between split datasets.
  • Figure 1 depicts one example system 100 for collaborative randomized encodings according to example aspects of the present disclosure.
  • the example system 100 contains a first computing system 102.
  • the first computing system 102 can be any type of system of one or more computing devices.
  • the first computing system can be a features computing system.
  • a computing device can be, for example, a personal computing device (e.g., laptop or desktop), a mobile computing device (e.g., smartphone or tablet), a gaming console or controller, a wearable computing device, an embedded computing device, a server computing device, a node of a distributed computing device, a virtual instance hosted on a shared server, or any other type of computing device.
  • the first computing system 102 includes a plurality of computing devices interconnected via a network or otherwise distributed in an interoperable manner.
  • the first computing system 102 can include a server for serving content over a network (e.g., network 104).
  • the first computing system 102 can include data 106.
  • Data 106 can include training data 108 or encoded data 114.
  • Training data 108 can include identifiers 110A-N and corresponding attributes 112A-N.
  • training data 108 can include pairs of ⁇ identifier, attribute ⁇ for N number of pairs.
  • Attributes 112A-N can include a number of attributes, such as features.
  • Features can include data representations of features associated with the identifiers such as an exposure to a content item, one or more preferences, characteristic data, or any other information. In some instances, the number of parameters can be on the order of millions of parameters associated with thousands of class labels.
  • Encoded data 114 can be generated by the first computing system 102 using a shared secret 116 and training data 108 as described herein.
  • the shared secret 116 can include a cryptographic key or set of cryptographic keys (e.g., a public key and a private key).
  • the shared secret 116 or shared secret 142 can be generated by shared secret generator 152.
  • Shared secret generator 152 can be a part of a server computing system 150.
  • the server computing system 150 can be part of the first computing system 102 or the second computing system 130.
  • the encoded data 114 can be stored for training machine-learned model 118. Using model trainer 122.
  • Example system 100 can include second computing system 130. In some implementations, second computing system 130 can be a labels computing system.
  • Second computing system 130 can store data training 132.
  • Training data 132 can include identifiers 136A-N and corresponding attributes 138A-N.
  • training data 132 can include pairs of ⁇ identifier, attribute ⁇ for N number of pairs.
  • Attributes 138A-N can include a number of attributes, such as labels.
  • training data 108 and training data 132 can include some overlap in identifiers such that for the subset of identifiers, training data 108 includes features and training data 134 includes labels.
  • Labels can include data representations of labels associated with the identifiers such as conversions actions performed, or other attributes which can be predicted based on the associated feature data. In some implementations, the labels can include thousands of class labels.
  • Encoded data 140 can be generated by the second computing system 130 using a shared secret 142 and training data 134 as described herein.
  • the shared secret 142 can include a cryptographic key or set of cryptographic keys (e.g., a public key and a private key).
  • the encoded data 140 can be encoded such that the training data is sealed into a computation such that after the key is deleted, the initial input training data 134 cannot be determined by the first computing system 102, however the first computing system 102 can perform the joint computation using encoded data 140 and encoded data 114.
  • the encoded data 140 can be stored for training machine-learned model 118.
  • Model trainer 122 can use methods described herein to train or update machine-learned model 118. For instance, model trainer 122 can use the methods described herein to determine a gradient for encoded data 114 and encoded data 140, obtained from second computing system 130, to use to update machine-learned weights 120.
  • Model trainer 122 can use methods described herein to train or update machine- learned model 118. For instance, model trainer 122 can use the methods described herein to determine a gradient for encoded data 114 and encoded data 140, obtained from second computing system 130, to use to update machine-learned weights 120.
  • FIG. 2 depicts a swim lane diagram 200 for utilizing collaborative randomized encodings where a single party is transmitting their respective encoded data.
  • the swim lane diagram can include a first computing system 205 and a second computing system 210.
  • the first computing system 205 and second computing system 210 can be associated with the same entity or different entities.
  • the first computing system 205 and second computing system 310 can be separate divisions of a single corporate entity or can be associated with distinct corporate entities.
  • the second computing system 210 can store a private dataset D2.
  • the private dataset D2 can include a plurality (e.g., thousands or millions) of data.
  • the data can include, for example, identifiers, such as device or account identifiers, and labels.
  • the first private dataset DI and the second private dataset D2 can include a plurality of common identifiers such that computing information relating to the overlap of the sets would provide a benefit to the first and second computing system. Due to privacy concerns, the parties can be prevented from sharing the datasets in their raw form. For instance, a desire to keep each dataset k-anonymous such that it cannot be known whether an identifier is within either the first private dataset DI or the second private dataset D2 (such that nothing can be learned about a single identifier).
  • the first computing system 205 and second computing system 210 can generate a shared key at operation 225.
  • the shared key can be computed by one of the computing systems and shared with the other computing system.
  • the shared key is generated by using a shared key protocol.
  • the shared key can be used by the respective computing systems to encrypt and seal the respective datasets.
  • the encryption methods used at operation 230 and 235 seal the respective datasets into a joint computation such that once the shared key is deleted, nothing can be learned about the initial private datasets, however the joint computation can be performed on the encrypted data to learn some aggregate information about the initial private datasets.
  • the first computing system can utilize the shared key to encode the private dataset to generate a first encoded dataset DI’.
  • the dataset is encoded using key encryption.
  • the plain text can be transformed into ciphertext using the shared key.
  • encrypting can include deriving a hash seed from the shared key and performing a hash function on the data.
  • the hashed data can be sorted into buckets.
  • encrypting DI can further include utilization of a garbled circuit as is described further with regard to Figure 3.
  • the first encoded dataset DI’ can be stored by the first computing system.
  • the first computing system can delete the shared key which effectively seals the encrypted data into the joint computation.
  • the shared key can be deleted in a number of ways. For instance, the computing systems can utilize secure erasure, key destruction, key management system protocol, or revocation.
  • the second computing system can utilize the shared key to encode the private dataset to generate a second encoded dataset D2’. Similar methods to those described at operation 230 can be used by second computing system 210 to generate the second encoded dataset D2’.
  • the second encoded dataset D2’ can be stored by the second computing system.
  • the first computing system and the second computing system can delete the shared key which effectively seals the encrypted data into the joint computation.
  • the second computing system can transmit the second encoded dataset D2’ to the first computing system.
  • the first computing system can make a call to the second computing system to request that the second encoded dataset D2’ be transmitted to the first computing system so that the joint computation can be evaluated for the encoded datasets.
  • the first computing system can generate an output by evaluating the fixed joint computation using the first encoded dataset DI’ and the second encoded dataset D2’ as input.
  • the fixed computation can include determining an intersection sum or training a machine-learned model that is held by one of the parties, as depicted in Figure 3.
  • Figure 3 depicts an example swim lane diagram 300 for training a machine- learned model using vertically split training data according to example embodiment of the present disclosure.
  • the swim lane diagram includes a first computing system 305 and a second computing system 310.
  • the first computing system 305 can be a features computing system storing identifiers and associated features data.
  • the second computing system 310 can be a labels computing system storing identifiers and associated labels data.
  • the first computing system 305 can, at operation 315, maintain a machine-learned model and store dataset.
  • the dataset can include a number of identifiers and features represented by ⁇ ID, x ⁇ .
  • the second computing system 310 can, at operation 320, store a dataset.
  • the dataset can include a number of identifiers and labels represented by ⁇ ID, y ⁇ .
  • the first computing system 305 and second computing system 310 can, at operation 325, perform an exchange of public keys for a batch of IDs to generate a key pair. The key pair is used by the respective computing systems to derive a hash seed at operation 330 and 335.
  • the first computing system 305 can generate a key pair comprising a first private key and a first public key.
  • the second computing system 310 can generate a key pair including a second private key and a second public key.
  • the first computing system 305 can share the first public key with the second computing system 310 and the second computing system 310 can share the second public key with the first computing system 305.
  • the joined key can be computed by the first computing system 305 using the first private key and the second public key.
  • the joined key can be computed by the second computing system 310 using the second private key and the first public key.
  • First computing system 305 can at operation 330, derive the hash seed and map the features and IDs to buckets represented by ⁇ bucket, ID, x ⁇ .
  • the first computing system 305 can perform cuckoo hashing such that each bucket is associated with at most a single feature record.
  • Second computing system 310 can at operation 335, derive the hash seed and map the labels to buckets represented by ⁇ bucket, (ID, y). . .(ID, y) ⁇ . Second computing system 310 can perform simple hashing such that each bucket can be associated with more than one label record. For instance, a single hashed bucket of the second encoded dataset can correspond to a plurality of buckets of the first encoded dataset.
  • the first computing system 305 can perform a number of operations for each bucket.
  • the first computing system 305 can generate a random vector, r.
  • the random vector, r is generated using the generated key pair such that the random vector generated by the first computing system can be generated by the second computing system (e.g., shared randomness).
  • the first computing system 305 can perform training of the machine-learned model using the random vector. For instance, the first computing system 305 can compute a noisy gradient for each bucket using the random vector. The first computing system 305 can sum the gradient for each bucket to generate the noisy gradient for the batch of IDs. The random vectors are used as masks for the respective buckets.
  • the number of features of the input datasets can be fixed.
  • the gradient update for a set of data can be represented as an affine function of the labels.
  • the gradient update for a batch of training data is the sum of the gradients in the batch. This can be represented by the function:
  • the gradient can be performed once using the masks (e.g., r), once using the masked labels (e.g., labels plus r), and subtract the results to get the actual gradient (including differentially private noise).
  • the features can be the features associated with private dataset DI
  • the labels can be the labels associated with private dataset D2
  • r can be a random value generated to provide differential privacy.
  • the gradient is an affine function of the labels such that:
  • C and D are constants that depend on the models and features. This formula can be used to compute the following:
  • r is a random mask that is applied to the labels.
  • Standard machine- learned models such as those built on ReLU, SoftMax, sigmoid, Poisson, linear embeddings, have the affine property when using a standard loss function such as Poisson loss, sigmoid loss, cross-entropy, or SoftMax-loss.
  • the first computing system and second computing system can generate random masks, r.
  • the first computing system can generate the Gradient of (feature, r) - noise locally.
  • the first computing system can generate input keys for the randomized encodings corresponding to its IDs.
  • the “value” of the garbled circuit is obtained as output when the gate includes the actual input value (e.g., there is a match between the actual input value and the input value used by the party computing the garbled circuit).
  • the “default” of the garbled circuit is obtained as output when there is not a match (e.g., the gate does not include the input value).
  • This method provides for safe randomized encoding using Gradr, wires, and garbled circuit. This is because if we assume that Gradr does not exist, GC and W together allow computing the masked labels in the intersection and nothing more. Since the labels are masked with random values that have been deleted and the value-or-default circuit hides whether there is a key (e.g., ID) in common between the two initial private datasets, the only data that can be revealed is something appears to be uniformly random (or statistically close to uniformly random). Thus, Gradr reveals nothing more than could be learned using Gradient(features, labels) plus noise and the masked labels.
  • the first computing system 305 can compute a gradient update based on the noisy gradient determined at operation 340B and stored at operation 350 and the masked labels gradient determined at operation 370.
  • the gradient update for the batch of IDs can be computed by taking two times the sum of the gradient for the masked labels minus the noisy gradient.
  • An update of the machine-learned model can be performed by adjusting the weights of machine-learned model.
  • the training can be performed with a regular cadence, such as once a day, once a week, or some other frequency. In some implementations, the training can be performed on demand.
  • the trained machine-learned model can then be used to predict labels (e.g., some sort of classification) based on input data including features. As the model is used to make predictions, the private datasets of the respective computing systems can be updated such that training can continue on real-time data.
  • Figure 4 depicts a swim lane diagram 400 for utilizing collaborative randomized encodings where both parties are transmitting their respective encoded data to compute a joint computation.
  • Figure 4 presents a similar approach to that described in Figure 2 but further elaborates on an implementation using a garbled circuit to seal the data as well as depicting an example embodiment involving the encoded data being shared by both computing systems.
  • the methods and embodiments described can be adjusted, combined, or altered such that a sub step described in one figure can be integrated with or replaced by sub steps in another feature. As such, the figures are for explanatory purposes only and are not meant to be limiting.
  • the swim lane diagram can include a first computing system 405 and a second computing system 410.
  • the first computing system 405 and second computing system 410 can be associated with the same entity or different entities.
  • the first computing system 405 can, at operation 415, store a private dataset DI.
  • the private dataset DI can include a plurality (e.g., thousands, millions, or billions) of data.
  • the data can include, for example, identifiers such as device or account identifiers and features.
  • the second computing system 410 can store a private dataset D2.
  • the private dataset D2 can include a plurality (e.g., thousands or millions) of data.
  • the data can include, for example, identifiers, such as device or account identifiers, and labels.
  • the first private dataset DI and the second private dataset D2 can include a plurality of common identifiers such that computing information relating to the overlap of the sets would provide a benefit to the first and second computing system.
  • the parties can be prevented from sharing the datasets in their raw form. For instance, a desire to keep each dataset k-anonymous such that it cannot be known whether an identifier is within either the first private dataset DI or the second private dataset D4.
  • the first computing system 405 and second computing system 410 can generate a shared key at operation 425.
  • the shared key can be computed by one of the computing systems and shared with the other computing system.
  • the shared key can be generated as described herein.
  • the shared key can be used by the respective computing systems to encrypt and seal the respective datasets.
  • the encryption methods used at operation 430 and 435 seal the respective datasets into a joint computation such that once the shared key is deleted, nothing can be learned about the initial private datasets, however the joint computation can be performed on the encrypted data to learn some aggregate information about the initial private datasets.
  • the first computing system can utilize the shared key to encode the private dataset to generate a first randomly encoded dataset.
  • a garbled circuit for the joint computation can be generated.
  • a garbled circuit can be generated to represent the joint computation.
  • a garbled circuit is a cryptographic protocol that allows for secure multi-party computation to allow for computing a function on private inputs without revealing the inputs to one another.
  • the garbled circuit consists of a circuit representation of the function to be computed. Each gate of the circuit is encrypted with the possible input combinations. The party with the function to be computed sends the garbled circuit to the other party.
  • the party sending the garbled circuit can provide its inputs by transmitting the corresponding encrypted values.
  • the receiving party that is computing the function can evaluate the garbled circuit gate by gate without learning the actual inputs from the transmitting party.
  • the function can be computed while preserving the privacy of the input data.
  • the garbled circuit can be utilized to generate a randomized encoding of the input data. For instance, for each gate of the garbled circuit, the computing system can retain only 1 out of the 2 wire keys for any input wire and forget if the input wire key corresponds to a 1 or 0.
  • the semantic meaning of the output wire key (e.g., which corresponds to 0 or 1) can be retained. Thus, only one path through the circuit can be decrypted and everything but the output (e.g., computation) appears random. This effectively fuses the input data into the computation and allows the input data to be forgotten. Thus, the only semantically useful thing that can be performed with the information shared (e.g., gates and input wires) is compute the final output.
  • the stored private dataset DI can be input into the garbled circuit.
  • the input wires associated with private dataset DI being processed by the garbled circuit can be stored as input wires W 1 along with the gates of the garbled circuit.
  • the first computing system 405 can store the input wires W 1 and the garbled gates. After the input wires W 1 and the garbled gates are stored, the first computing system 405 can delete the shared key.
  • the first private dataset has been sealed into the fixed computation represented by the gate.
  • the input wires W 1 can be a set of input wires associated with the respective pairs of data of the private dataset DI.
  • the second computing system can utilize the shared key to encode the private dataset to generate a second randomly encoded dataset. For instance, a garbled circuit for the joint computation can be generated (e.g., as described with regard to operation 430).
  • the joint computation can be a value-or-default circuit as described herein.
  • Value-or-default can be computed efficiently using hash-to-buckets for the input data.
  • the keys of the key-value pairs can be hashed into buckets.
  • the system can pad the buckets such that each bucket is equal in size.
  • the value-or-default function can be performed bucket-by-bucket. By hashing the values, the buckets can be relatively small which allows the value-or-default circuit to be relatively small and allow for faster computation of the value-or-default between the encoded datasets.
  • the values associated with the keys can be randomized.
  • the randomization of the values must be compatible with the aggregated functionality (e.g., joint computation) such that the output of the joint computation is useful (e.g., for updating the weights of the machine-learned model or performing an intersection sum).
  • the stored private dataset D2 can be input into the garbled circuit.
  • the input wires associated with private dataset D2 being processed by the garbled circuit can be stored as input wires W2 along with the gates of the garbled circuit.
  • the second computing system 410 can store the input wires W2 and the garbled gates. After the input wires W2 and the garbled gates are stored, the second computing system 410 can delete the shared key.
  • the first private dataset has been sealed into the fixed computation represented by the gate.
  • the input wires W2 can be a set of input wires associated with the respective pairs of data of the private dataset D2.
  • the first computing system and the second computing system can delete the shared key and seal the encoded data into the joint computation.
  • the first computing system can transmit the first encoded dataset comprising the W1 input wires to the second computing system.
  • the second computing system can make a call to the first computing system to request that the first encoded dataset be transmitted to the second computing system so that the joint computation can be evaluated for the encoded datasets.
  • the second computing system can transmit the second encoded dataset comprising the W2 input wires to the first computing system.
  • the first computing system can make a call to the second computing system to request that the second encoded dataset be transmitted to the first computing system so that the joint computation can be evaluated for the encoded datasets.
  • the first computing system can generate an output by evaluating the garbled circuit over W1 and W2. For instance, the parties can perform a join and compute based on W1 and W2. For instance, each party holds key -value pairs.
  • the garbled circuit can be evaluated to compute the aggregated function of the values for keys that match. Thus, the final noisy output is revealed without the ability for either system to determine whether any particular key matched. This can be performed using a garbled circuit representing a value or default function.
  • the second computing system can generate an output by evaluating the garbled circuit over W1 and W2.
  • the output can be used to perform an intersection of sums, determine a gradient (e.g., as described in Figure 3), or otherwise utilized to perform the joint computation associated with the garbled circuit.
  • FIG. 5 depicts a flow diagram of an example method 500 to perform secure multi-party computation in accordance with some embodiments of the present disclosure.
  • the method 500 can be performed by processing logic that can include hardware (e.g., processing device, circuitry, dedicated logic, programmable logic, microcode, hardware of a device, integrated circuit, etc.), software (e.g., instructions run or executed on a processing device), or a combination thereof.
  • method 500 is performed by a server computing system (e.g., server computing system 704) or a client computing system (e.g., first computing system 702 or second computing system 708).
  • server computing system e.g., server computing system 704
  • client computing system e.g., first computing system 702 or second computing system 708
  • the illustrated embodiments should be understood only as examples, and the illustrated processes can be performed in a different order, and some processes can be performed in parallel. Additionally, one or more processors can be omitted in various embodiments. Thus, not all processes are required in every embodiment. Other process flows are possible.
  • processing logic can maintain, by the first computing system, a machine-learned model.
  • the machine-learned model is built on at least one of ReLU layers, SoftMax layers, sigmoid layers, Poisson layers, or linear embeddings.
  • the machine-learned model can be a classification model.
  • processing logic can generate a first encoded dataset by sealing, by the first computing system using a shared key, a first private dataset into a fixed joint computation.
  • the first private dataset can include a plurality of identifiers and associated features and the second private dataset comprises a plurality of identifiers and associated labels and the first private dataset and second private dataset can include a plurality of common identifiers.
  • the joint computation can include a garbled circuit representing a value-or-default function. Generating the encoded dataset can include one path through the garbled circuit that can be decrypted by, for each input wire, retaining one wire key and forgetting if the wire key corresponds to 1 or 0; and retaining a semantic meaning of an output wire for a respective gate.
  • the fixed joint computation can include an intersection sum.
  • the first encoded dataset can be generated by hashing the first private dataset into buckets.
  • the processing logic can generate a plurality of buckets using a hash seed derived from the shared secret, wherein the buckets comprise a bucket identifier, a hashed identifier, and a hashed feature, wherein the buckets are generated using cuckoo hashing such that each bucket identifier is associated with at most one identifier of the first private dataset.
  • the processing logic can perform operations. For instance, the processing logic can generate a random vector. The processing logic can determine an error by performing backpropagation of the machine-learned model using the input comprising the hashed feature and the random vector. The processing logic can compute a garbled circuit. The processing logic can store wire labels of the garbled circuit for the respective bucket. In some instances, the sum of the random vectors can equal zero. In some implementations, the sum of the random vectors can equal a randomly generated number (e.g., an amount of differentially private noise). [0094] The shared key can be generated by the first computing system performing a public key exchange with the second computing system. The shared key can be generated by the first computing system and shared with the second computing system.
  • processing logic can delete the shared key.
  • the second encoded dataset can be generated by hashing the second private dataset into buckets.
  • the processing logic can generate buckets using the hash seed derived from the shared secret, wherein the buckets comprise a bucket identifier, hashed identifiers, and one or more masked labels, wherein the buckets are generated using standard hashing such that each bucket identifier can be associated with one or more masked labels, wherein the labels are masked by adding a random vector to the label.
  • processing logic can generate an output by evaluating the fixed joint computation using the first encoded dataset and the second encoded dataset as input.
  • processing logic can update the machine-learned model based on the output.
  • the machine-learned model can include a number of layers with weighted parameters. Updating the machine-learned model can include updating the parameters based on a gradient associated with the training data.
  • the machine-learned model is built on at least one of ReLU layers, SoftMax layers, sigmoid layers, Poisson layers, or linear embeddings.
  • the updated machine-learned model can be used to make predictions based on obtained input data.
  • the processing logic can obtain a new dataset including a number of identifier-feature sets.
  • the identifier-feature sets can include an identifier and one or more features associated with the identifier.
  • the processing logic can input the new dataset into the updated machine-learned model.
  • the processing logic can obtain output including a prediction for a label for each identifier-feature set of the plurality of identifier-feature sets.
  • Figure 6 A and Figure 6B depict a flow diagram of an example method 600 to perform content load and placement in accordance with some embodiments of the present disclosure.
  • the method 600 can be performed by processing logic that can include hardware (e.g., processing device, circuitry, dedicated logic, programmable logic, microcode, hardware of a device, integrated circuit, etc.), software (e.g., instructions run or executed on a processing device), or a combination thereof.
  • processing logic can include hardware (e.g., processing device, circuitry, dedicated logic, programmable logic, microcode, hardware of a device, integrated circuit, etc.), software (e.g., instructions run or executed on a processing device), or a combination thereof.
  • method 600 is performed by a server computing system (e.g., server computing system 704) or a client computing system (e.g., first computing system 702 or second computing system 708).
  • server computing system e.g., server computing system 704
  • client computing system e.g., first computing system 702 or second computing system 708
  • the illustrated embodiments should be understood only as examples, and the illustrated processes can be performed in a different order, and some processes can be performed in
  • processing logic can maintain a machine-learned model comprising a plurality of weights.
  • the machine-learned model is built on at least one of ReLU layers, SoftMax layers, sigmoid layers, Poisson layers, or linear embeddings.
  • the machine-learned model can be a classification model.
  • processing logic can generate, by a first computing system, using input data comprising a batch of a first private dataset and a hash seed derived from a shared secret, a plurality of buckets of data comprising a bucket ID, a hashed identifier, and a hashed feature. For instance, a batch of a first private dataset can be selected based on a batch identifier and batch size determined for use for training the machine-learned model. The batch identifier can be used to determine a set of training data that includes common identifiers between a first private dataset and a second private dataset.
  • the computing systems can generate a shared secret.
  • the shared secret can be a cryptographic key as described herein.
  • the shared secret can be utilized to derive a hash seed which can be used to generate buckets. Bucketing the datasets can allow for faster processing in later steps performed by the processing logic.
  • processing logic can generate, by the first computing system, a first set of random encoded data of the batch of the first private dataset by for each bucket of the plurality of buckets: generating a random vector; determining an error by performing backpropagation of the machine-learned model using input comprising the hashed feature and the random vector; and computing a garbled circuit.
  • the random vector can be a number of values generated using the shared secret.
  • the processing logic can determine a noisy gradient for the machine-learned model using the random vector as described with regard to Figure 3.
  • processing logic can generate, by the first computing system, a first sum of errors by summing the errors of each bucket of the plurality of buckets to determine a noisy sum of errors.
  • the sum of errors can be a noisy gradient for the machine-learned model.
  • the noisy gradient can be determined for each bucket of the plurality of buckets.
  • the overall noisy gradient can be determined by summing the noisy gradient determined for each bucket.
  • processing logic can store, by the first computing system: (i) the sum of errors, (ii) public key, and (iii) a plurality of stored buckets comprising: the garbled circuit for each bucket, the wire labels for each bucket.
  • the first computing system can store the data needed to compute the joint function once the second encoded dataset is received from the second computing system.
  • processing logic can delete the shared secret. For instance, any number of cryptographic deletion techniques can be used. In some instances, the secret can be located and be written over with random data to make the key incapable of being recovered. Any existing techniques can be used to delete the shared secret.
  • processing logic can obtain a second set of random encoded data comprising (i) encrypted masked labels for a plurality of buckets and (ii) wires associated with the encrypted masked labels for a plurality of buckets.
  • the encrypted masked labels for the plurality of buckets include a label plus random vector which is then input into a garbled circuit.
  • the input wires associated with the encrypted masked labels are stored by the second computing system and the shared secret is deleted. Following the deletion of the shared secret, the second set of random encoded data is transmitted from the second computing system to the first computing system.
  • processing logic can, for each bucket of the plurality of stored buckets: evaluate the garbled circuit to generate a recovered masked label as output; and determine an error by performing backpropagation of the machine-learned model using input comprising a feature and the recovered masked label.
  • determining the error by performing backpropagation of the machine-learned model is performed using input comprising a feature and the recovered masked label divided by two.
  • the masked label can be divided by a factor. In some instances, the optimal factor can be two.
  • the garbled circuit can be a representation of a value-or- default function such that if a key is in common between the first encoded dataset and the second encoded dataset, the masked label will be returned and if there is no match, the default label value plus the random vector will be returned.
  • the error determined can be a gradient of the masked labels such that the gradient of the features and labels can be calculated as described herein.
  • Backpropagation can be performed using a standard loss function.
  • the standard loss function can include at least one of Poisson loss, sigmoid loss, cross-entropy, or SoftMax-loss.
  • processing logic can compute, by the first computing system, a gradient update for the batch.
  • the gradient update for the batch is computed by doubling the sum of the second set of errors minus the first sum of errors.
  • the gradient update is computed based on a factor of the sum of the second set of errors. In some implementations, an optimal factor can be two.
  • the gradient update can be generated by combining the first sum of errors and the second sum of errors.
  • the first sum of errors can be indicative of the noisy gradient.
  • the second sum of errors can be indicative of the masked labels gradient.
  • the calculated gradients can be used to determine the gradient update which can in turn be used to update the machine-learned model.
  • processing logic can update, by the first computing system, the plurality of weights associated with the machine-learned model based on the gradient update.
  • the machine-learned model can be continually trained by repeating the process for multiple batches of training data which can be gathered in a single time or can be gathered over a number of days. For instance, gradient updates can be performed using parallel processing allowing for training with multiple datasets to be performed simultaneously. Additionally, or alternatively, the machine-learned model can be trained and updated on a regular cadence such that trends in data can be taken into account for future predictions made by the model. Thus, the machine-learned model can be updated in near-real time.
  • FIG. 7 depicts a block diagram of an example computing system 700 that performs collaborative randomized encodings according to example embodiments of the present disclosure.
  • the computing system 700 includes a first computing system 702, a server computing system 704, a training computing system 706, and a second computing system 708 that are communicatively coupled over a network 730.
  • the first computing system 702 can be any type of computing device, such as, for example, a personal computing device (e.g., laptop or desktop), a mobile computing device (e.g., smartphone or tablet), a gaming console or controller, a wearable computing device, an embedded computing device, or any other type of computing device.
  • a personal computing device e.g., laptop or desktop
  • a mobile computing device e.g., smartphone or tablet
  • a gaming console or controller e.g., a gaming console or controller
  • a wearable computing device e.g., an embedded computing device, or any other type of computing device.
  • the first computing system 702 includes one or more processors 712 and a memory 714.
  • the one or more processors 712 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected.
  • the memory 714 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof.
  • the memory 714 can store data 716 and instructions 718 which are executed by the processor 712 to cause the first computing system 702 to perform operations.
  • the client computing system can include a user interface.
  • the user interface can include a graphical user interface, audio user interface, touch user interface, or any other user interface.
  • the client computing system can include a user input component.
  • the user input component can be associated with user interface and can be capable of obtaining user input.
  • user input can include touch, audio, or other user input.
  • user input components can be capable of obtaining user input and translating the user input into a computer readable form.
  • First computing system 702 can include database 722.
  • Database 722 can store private data 724 and encoded data 726. Private data 724 can be stored by first computing system 702.
  • Encoded data 726 can be stored by first computing system 702 and transmitting to second computing system 708 after a shared key has been destroyed by second computing system 708.
  • the first computing system 702 can store or otherwise include one or more models 720.
  • the models 720 can be or can otherwise include various statistical or machine-learned models.
  • Example machine-learned models include neural networks or other multi-layer non-linear models.
  • Example neural networks include feed forward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks.
  • Some example machine-learned models can leverage an attention mechanism such as self-attention.
  • some example machine-learned models can include multi-headed self-attention models (e.g., transformer models).
  • the first computing system 702, second computing system 708, or the server computing system 704 can train the models 720, 740, 790 via interaction with the training computing system 706 that is communicatively coupled over the network 730.
  • the training computing system 706 can be separate from the first computing system 702, second computing system 708, or server computing system 704 or can be a portion of the first computing system 702, second computing system 708, or server computing system 704.
  • the server computing system 704 includes one or more processors 732 and a memory 734.
  • the one or more processors 732 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected.
  • the memory 634 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof.
  • the memory 734 can store data 736 and instructions 738 which are executed by the processor 732 to cause the server computing system 704 to perform operations.
  • the server computing system 704 includes or is otherwise implemented by one or more server computing devices. In instances in which the server computing system 704 includes plural server computing devices, such server computing devices can operate according to sequential computing architectures, parallel computing architectures, or some combination thereof.
  • Server computing system 704 can be configured to obtain data from first computing system 702 (e.g., via an application) or second computing system 708. For instance, server computing system 704 can utilize the obtained user input data to update or train one or more models 720, 740, or 790.
  • the server computing system 704 can store or otherwise include one or more models 740.
  • the models 740 can be or can otherwise include various statistical or machine-learned models.
  • Example machine-learned models include neural networks or other multi-layer non-linear models.
  • Example neural networks include feed forward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks.
  • Some example machine-learned models can leverage an attention mechanism such as self-attention.
  • some example machine-learned models can include multi-headed self-attention models (e.g., transformer models).
  • the second computing system 708 includes one or more processors 772 and a memory 774.
  • the one or more processors 772 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected.
  • the memory 774 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof.
  • the memory 774 can store data 776 and instructions 778 which are executed by the processor 772 to cause the second computing system 708 to perform operations.
  • the second computing system 708 includes or is otherwise implemented by one or more server computing devices.
  • server computing devices can operate according to sequential computing architectures, parallel computing architectures, or some combination thereof.
  • Second computing system 708 can include database 780.
  • Database 780 can store private data 781 and encoded data 782. Private data 781 can be stored by second computing system 708. Encoded data 782 can be stored by second computing system 708 and transmitting to first computing system 702 after a shared key has been destroyed by first computing system 702.
  • Second computing system 708 can be communicatively connected over network 730 to server computing system 704.
  • second computing system 708 can be a computing system associated with the server computing system 704 or first computing system 702.
  • second computing system 708 can be associated with a third- party, for instance, a content provider (e.g., advertiser). There can be more than one second computing system 708.
  • a content provider e.g., advertiser
  • the training computing system 706 includes one or more processors 752 and a memory 754.
  • the one or more processors 752 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected.
  • the memory 754 can include one or more non -transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof.
  • the memory 754 can store data 756 and instructions 758 which are executed by the processor 752 to cause the training computing system 706 to perform operations.
  • the training computing system 706 includes or is otherwise implemented by one or more server computing devices.
  • the training computing system 706 can include a model trainer 760 that trains the machine-learned models 720, 740, or 790 stored at the first computing system 702, the server computing system 704, or the second computing system 708 using various training or learning techniques, such as, for example, backwards propagation of errors.
  • a loss function can be backpropagated through the model(s) to update one or more parameters of the model(s) (e.g., based on a gradient of the loss function).
  • Various loss functions can be used such as mean squared error, likelihood loss, cross entropy loss, hinge loss, or various other loss functions.
  • Gradient descent techniques can be used to iteratively update the parameters over a number of training iterations. Gradient descent techniques are discussed in further detail with regard to Figure 3, Figure 5, and Figure 6.
  • performing backwards propagation of errors can include performing truncated backpropagation through time.
  • the model trainer 760 can perform a number of generalization techniques (e.g., weight decays, dropouts, etc.) to improve the generalization capability of the models being trained.
  • the model trainer 760 can train the models 720, 740, or 790 based on a set of training data.
  • the training data can include, for example, encoded data 782, encoded data 726, historic signal data, publisher-rendered native content item data, user input data, conversion data, user device location data, click data, or any other relevant data (e.g., data stored in database 780 or database 722, and the like).
  • the model trainer 760 includes computer logic utilized to provide desired functionality.
  • the model trainer 760 can be implemented in hardware, firmware, or software controlling a general -purpose processor.
  • the model trainer 760 includes program files stored on a storage device, loaded into a memory, and executed by one or more processors.
  • the model trainer 760 includes one or more sets of computer-executable instructions that are stored in a tangible computer- readable storage medium such as RAM, hard disk, or optical or magnetic media.
  • the network 730 can be any type of communications network, such as a local area network (e.g., intranet), wide area network (e.g., Internet), or some combination thereof and can include any number of wired or wireless links.
  • communication over the network 730 can be carried via any type of wired or wireless connection, using a wide variety of communication protocols (e.g., TCP/IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), or protection schemes (e.g., VPN, secure HTTP, SSL).
  • the input to the machine-learned model(s) of the present disclosure can be image data.
  • the machine-learned model(s) can process the image data to generate an output.
  • the machine-learned model(s) can process the image data to generate an image recognition output (e.g., a recognition of the image data, a latent embedding of the image data, an encoded representation of the image data, a hash of the image data, etc.).
  • the machine-learned model(s) can process the image data to generate an image segmentation output.
  • the machine-learned model(s) can process the image data to generate an image classification output.
  • the machine-learned model(s) can process the image data to generate an image data modification output (e.g., an alteration of the image data, etc.).
  • the machine-learned model(s) can process the image data to generate an encoded image data output (e.g., an encoded and/or compressed representation of the image data, etc.).
  • the machine-learned model(s) can process the image data to generate an upscaled image data output.
  • the machine-learned model(s) can process the image data to generate a prediction output.
  • the input to the machine-learned model(s) of the present disclosure can be text or natural language data.
  • the machine-learned model(s) can process the text or natural language data to generate an output.
  • the machine-learned model(s) can process the natural language data to generate a language encoding output.
  • the machine-learned model(s) can process the text or natural language data to generate a latent text embedding output.
  • the machine-learned model(s) can process the text or natural language data to generate a translation output.
  • the machine-learned model(s) can process the text or natural language data to generate a classification output.
  • the machine-learned model(s) can process the text or natural language data to generate a textual segmentation output.
  • the machine-learned model(s) can process the text or natural language data to generate a semantic intent output.
  • the machine-learned model(s) can process the text or natural language data to generate an upscaled text or natural language output (e.g., text or natural language data that is higher quality than the input text or natural language, etc.).
  • the machine-learned model(s) can process the text or natural language data to generate a prediction output.
  • the input to the machine-learned model(s) of the present disclosure can be speech data.
  • the machine-learned model(s) can process the speech data to generate an output.
  • the machine-learned model(s) can process the speech data to generate a speech recognition output.
  • the machine-learned model(s) can process the speech data to generate a speech translation output.
  • the machine-learned model(s) can process the speech data to generate a latent embedding output.
  • the machine-learned model(s) can process the speech data to generate an encoded speech output (e.g., an encoded or compressed representation of the speech data, etc.).
  • an encoded speech output e.g., an encoded or compressed representation of the speech data, etc.
  • the machine-learned model(s) can process the speech data to generate an upscaled speech output (e.g., speech data that is higher quality than the input speech data, etc.).
  • the machine-learned model(s) can process the speech data to generate a textual representation output (e.g., a textual representation of the input speech data, etc.).
  • the machine-learned model(s) can process the speech data to generate a prediction output.
  • the input to the machine-learned model(s) of the present disclosure can be latent encoding data (e.g., a latent space representation of an input, etc.).
  • the machine-learned model(s) can process the latent encoding data to generate an output.
  • the machine-learned model(s) can process the latent encoding data to generate a recognition output.
  • the machine-learned model(s) can process the latent encoding data to generate a reconstruction output.
  • the machine-learned model(s) can process the latent encoding data to generate a search output.
  • the machine-learned model(s) can process the latent encoding data to generate a reclustering output.
  • the machine-learned model(s) can process the latent encoding data to generate a prediction output.
  • the input to the machine-learned model(s) of the present disclosure can be statistical data.
  • Statistical data can be, represent, or otherwise include data computed or calculated from some other data source.
  • the machine-learned model(s) can process the statistical data to generate an output.
  • the machine-learned model(s) can process the statistical data to generate a recognition output.
  • the machine-learned model(s) can process the statistical data to generate a prediction output.
  • the machine-learned model(s) can process the statistical data to generate a classification output.
  • the machine-learned model(s) can process the statistical data to generate a segmentation output.
  • the machine-learned model(s) can process the statistical data to generate a visualization output.
  • the machine-learned model(s) can process the statistical data to generate a diagnostic output.
  • the input includes visual data
  • the task is a computer vision task.
  • the input includes pixel data for one or more images and the task is an image processing task.
  • the image processing task can be image classification, where the output is a set of scores, each score corresponding to a different object class and representing the likelihood that the one or more images depict an object belonging to the object class.
  • the image processing task may be object detection, where the image processing output identifies one or more regions in the one or more images and, for each region, a likelihood that region depicts an object of interest.
  • the image processing task can be image segmentation, where the image processing output defines, for each pixel in the one or more images, a respective likelihood for each category in a predetermined set of categories.
  • the set of categories can be foreground and background.
  • the set of categories can be object classes.
  • the image processing task can be depth estimation, where the image processing output defines, for each pixel in the one or more images, a respective depth value.
  • the image processing task can be motion estimation, where the network input includes multiple images, and the image processing output defines, for each pixel of one of the input images, a motion of the scene depicted at the pixel between the images in the network input.
  • the input includes audio data representing a spoken utterance and the task is a speech recognition task.
  • the output may comprise a text output which is mapped to the spoken utterance.
  • the task comprises encrypting or decrypting input data.
  • the task comprises a microprocessor performance task, such as branch prediction or memory address translation.
  • Figure 7 illustrates one example computing system that can be used to implement the present disclosure.
  • the first computing system 702 or second computing system 708 can include the model trainer 760 and the training data.
  • the models 720, 740, or 790 can be both trained and used locally at the first computing system 702 or second computing system 708.
  • the technology discussed herein makes reference to servers, databases, software applications, and other computer-based systems, as well as actions taken, and information sent to and from such systems.
  • the inherent flexibility of computer-based systems allows for a great variety of possible configurations, combinations, and divisions of tasks and functionality between and among components. For instance, processes discussed herein can be implemented using a single device or component or multiple devices or components working in combination. Databases and applications can be implemented on a single system or distributed across multiple systems. Distributed components can operate sequentially or in parallel.
  • the functions or steps described herein can be embodied in computer-usable data or computer-executable instructions, executed by one or more computers or other devices to perform one or more functions described herein.
  • data or instructions include routines, programs, objects, components, data structures, or the like that perform particular tasks or implement particular data types when executed by one or more processors in a computer or other data-processing device.
  • the computer-executable instructions can be stored on a computer-readable medium such as a hard disk, optical disk, removable storage media, solid-state memory, read-only memory (ROM), random-access memory (RAM), or the like.
  • ROM read-only memory
  • RAM random-access memory
  • the functionality can be embodied in whole or in part in firmware or hardware equivalents, such as integrated circuits, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or the like.
  • firmware or hardware equivalents such as integrated circuits, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or the like.
  • Particular data structures can be used to implement one or more aspects of the disclosure more effectively, and such data structures are contemplated to be within the scope of computer-executable instructions or computer-usable data described herein.
  • aspects described herein can be embodied as a method, system, apparatus, or one or more computer-readable media storing computer-executable instructions. Accordingly, aspects can take the form of an entirely hardware embodiment, an entirely software embodiment, an entirely firmware embodiment, or an embodiment combining software, hardware, or firmware aspects in any combination.
  • the various methods and acts can be operative across one or more computing devices or networks.
  • the functionality can be distributed in any manner or can be located in a single computing device (e.g., server, client computer, user device, or the like).

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Storage Device Security (AREA)

Abstract

Des modes de réalisation donnés à titre d'exemple de la présente divulgation concernent un procédé donné à titre d'exemple pour un codage randomisé collaboratif permettant d'entraîner des modèles appris par machine avec des données d'entraînement divisées verticalement. Le procédé peut consister à maintenir un modèle appris par machine. Le procédé peut également consister à générer un premier ensemble de données codées par scellement, à l'aide d'une clé partagée, d'un premier ensemble de données privées en un calcul conjoint fixe. Le procédé peut de même consister à effectuer un stockage, par le premier ensemble de données codées. Le procédé peut aussi consister à supprimer la clé partagée. Le procédé peut par ailleurs consister à obtenir un second ensemble de données codées de la part d'un second système informatique. Le second ensemble de données codées a été généré par le second système informatique à l'aide de la clé partagée pour sceller un second ensemble de données privé dans le calcul conjoint fixe. Le procédé peut en outre consister à générer une sortie par évaluation du calcul conjoint fixe à l'aide du premier ensemble de données codées et du second ensemble de données codées en tant qu'entrée et à mettre à jour le modèle appris par machine sur la base de la sortie.
PCT/US2024/035818 2023-12-29 2024-06-27 Codage randomisé collaboratif permettant d'entraîner des modèles appris par machine avec des données d'entraînement divisées verticalement Pending WO2025144460A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP24745567.8A EP4599421A1 (fr) 2023-12-29 2024-06-27 Codage randomisé collaboratif permettant d'entraîner des modèles appris par machine avec des données d'entraînement divisées verticalement

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202363616445P 2023-12-29 2023-12-29
US63/616,445 2023-12-29

Publications (1)

Publication Number Publication Date
WO2025144460A1 true WO2025144460A1 (fr) 2025-07-03

Family

ID=91959456

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2024/035818 Pending WO2025144460A1 (fr) 2023-12-29 2024-06-27 Codage randomisé collaboratif permettant d'entraîner des modèles appris par machine avec des données d'entraînement divisées verticalement

Country Status (2)

Country Link
EP (1) EP4599421A1 (fr)
WO (1) WO2025144460A1 (fr)

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018174873A1 (fr) * 2017-03-22 2018-09-27 Visa International Service Association Apprentissage-machine de protection de la vie privée

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018174873A1 (fr) * 2017-03-22 2018-09-27 Visa International Service Association Apprentissage-machine de protection de la vie privée

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
BLASS ERIK-OLIVER ET AL: "Private Collaborative Data Cleaning via Non-Equi PSI", 2023 IEEE SYMPOSIUM ON SECURITY AND PRIVACY (SP), IEEE, 21 May 2023 (2023-05-21), pages 1419 - 1434, XP034380758, DOI: 10.1109/SP46215.2023.10179337 *
GLOVA ALVIN OLIVER ET AL: "Establishing Cooperative Computation with Hardware Embassies", 2022 IEEE INTERNATIONAL SYMPOSIUM ON SECURE AND PRIVATE EXECUTION ENVIRONMENT DESIGN (SEED), IEEE, 26 September 2022 (2022-09-26), pages 85 - 96, XP034220432, DOI: 10.1109/SEED55351.2022.00015 *

Also Published As

Publication number Publication date
EP4599421A1 (fr) 2025-08-13

Similar Documents

Publication Publication Date Title
Wang et al. A privacy-enhanced retrieval technology for the cloud-assisted internet of things
Naidu et al. A Robust medical data sharing application with a collaborative hypothesis and Elastic net regression model
US11354539B2 (en) Encrypted data model verification
US11222138B2 (en) Privacy-preserving machine learning in the three-server model
Soykan et al. A survey and guideline on privacy enhancing technologies for collaborative machine learning
CN110245510A (zh) 用于预测信息的方法和装置
Fritchman et al. Privacy-preserving scoring of tree ensembles: A novel framework for AI in healthcare
Selvi et al. Medical image encryption and compression by adaptive sigma filterized synorr certificateless signcryptive Levenshtein entropy-coding-based deep neural learning
CN110059501B (zh) 一种基于差分隐私的安全外包机器学习方法
Mehnaz et al. A secure sum protocol and its application to privacy-preserving multi-party analytics
Cheung et al. Fedsgc: Federated simple graph convolution for node classification
CN113055153B (zh) 一种基于全同态加密算法的数据加密方法、系统和介质
Ibarrondo et al. Banners: Binarized neural networks with replicated secret sharing
Amaithi Rajan et al. QMedShield: a novel quantum chaos-based image encryption scheme for secure medical image storage in the cloud
Zhang et al. An Efficient FHE-Enabled Secure Cloud–Edge Computing Architecture for IoMT Data Protection with its Application to Pandemic Modeling
Adamsetty et al. Securing machine learning models: Homomorphic encryption and its impact on classifiers
Yang et al. Privacy-preserving machine learning in cloud-edge-end collaborative environments
Abou Harb et al. Efficient Estimation of Sigmoid and Tanh Activation Functions for Homomorphically Encrypted Data Using Artificial Neural Networks
WO2025144460A1 (fr) Codage randomisé collaboratif permettant d'entraîner des modèles appris par machine avec des données d'entraînement divisées verticalement
Sumathi et al. Scale-based secured sensitive data storage for banking services in cloud
Sheikh et al. Secure multiparty computation: From millionaires problem to anonymizer
Nishida et al. Efficient secure neural network prediction protocol reducing accuracy degradation
Jia et al. Towards privacy-preserving and efficient word vector learning for lightweight IoT devices
Abdullah Survey: Privacy-Preserving in Deep Learning based on Homomorphic Encryption
Xu Functional encryption based approaches for practical privacy-preserving machine learning

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2024745567

Country of ref document: EP

Effective date: 20241230

WWP Wipo information: published in national office

Ref document number: 2024745567

Country of ref document: EP