[go: up one dir, main page]

WO2024253705A1 - Équité d'apprentissage automatique avec attributs protégés limités - Google Patents

Équité d'apprentissage automatique avec attributs protégés limités Download PDF

Info

Publication number
WO2024253705A1
WO2024253705A1 PCT/US2023/086020 US2023086020W WO2024253705A1 WO 2024253705 A1 WO2024253705 A1 WO 2024253705A1 US 2023086020 W US2023086020 W US 2023086020W WO 2024253705 A1 WO2024253705 A1 WO 2024253705A1
Authority
WO
WIPO (PCT)
Prior art keywords
dataset
model
protected
protected attributes
attributes
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/US2023/086020
Other languages
English (en)
Inventor
Rashidul Islam
Yiwei CAI
Hao Yang
Md Taufeeq UDDIN
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Visa International Service Association
Original Assignee
Visa International Service Association
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Visa International Service Association filed Critical Visa International Service Association
Publication of WO2024253705A1 publication Critical patent/WO2024253705A1/fr
Anticipated expiration legal-status Critical
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/20—Ensemble learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/0895—Weakly supervised learning, e.g. semi-supervised or self-supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/091—Active learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N7/00—Computing arrangements based on specific mathematical models
    • G06N7/01—Probabilistic graphical models, e.g. probabilistic networks

Definitions

  • a robust dataset allow for analysis of the machine learning models to understand if a trained model is biased against certain values within the datapoints. For instance, a choice of machine learning model or technique may lead to bias against a certain population within the dataset while a different choice may lead to bias against a different population.
  • a model or technique is trained, it is possible to evaluate based on the dataset what biases may exist. However, this approach assumes that all information is available about the underlying dataset on which training occurred.
  • the growing ubiquity of machine learning models in various aspects of decision making has brought fairness in machine learning in focus. For instance, various decisions, such as credit availability, loan approvals, hiring, and even criminal sentencing can rely on machine learning.
  • Fairness is machine learning can refer to the process of eliminating or correcting bias which may exist in a machine learning model to prevent or correct outcomes which may exist from any characteristic related to the data. For example, there may be an emergent or inherent bias which can be related to race, ethnicity, gender, sexual orientation, disability, veteran status, or other characteristic which may be related to the data. Thus, ensuring fairness in machine learning models is essential for promoting the positive impacts of 1 KILPATRICK TOWNSEND 780676821 machine learning while mitigating the negative impacts of machine learning on individuals, entities, and society.
  • PAs protected attributes
  • Embodiments disclosed include an upstream model that can learn predictive models for protected attributes (PAs) using limited data and then synthesize new PAs for use in downstream target tasks.
  • the estimated PAs can effectively replace original PAs, enabling the development of fair machine learning models that operate on limited access to PAs. Further, selecting high-quality examples and processing them with fairness in mind can help train competitive fair models from relatively small data sets.
  • Provided herein are systems, methods, and computer readable media that can achieve fairness in machine learning applications while using limited protected attributes.
  • a two-stage (a first stage (also referred to as an upstream stage, PA estimation stage, or bias estimation stage) and a second stage (also referred to as a downstream stage, fair model generation stage, or downstream fairness stage)) framework can leverage active learning techniques to predict protected attributes.
  • the framework can utilize these predictions, which can be based on limited labeled data, into a machine learning training process to achieve fairness in a target task.
  • the first stage (upstream) can employ active learning to accurate predict protect attributes using limited labeled data.
  • the second stage (downstream) can incrementally 2 KILPATRICK TOWNSEND 780676821 sample low-biased training examples for fair learning based on estimates provided from first stage.
  • sample selection strategies can be used to sample data from the available set of data. Sample selection strategies can further increase fairness while only accessing a limited number of PAs.
  • Embodiments disclosed include a method for correcting algorithmic bias in a machine learning framework.
  • the method may comprise obtaining, a first dataset, the first dataset containing protected attributes from at least one category of protected attributes; obtaining, a second dataset, the second dataset not containing any protected attributes from the at least one category of protected attributes; training, using the first dataset, a protected attributes estimation model, wherein the protected attributes estimation model estimates information related to the least one protected attribute; and predicting, using protected attributes estimation model, at least one protected attribute for data within the second dataset.
  • the method may further include any combination of the following aspects.
  • the protected attributes estimation model is trained in iterations. Each iteration may involve an active learning process to improve the protected attributes estimation model at each iteration. Additional datapoints containing protected attributes may be added to the first dataset to update the training of the protected attributes.
  • Additional datapoints may be obtained by annotating points from the second dataset with a protected attribute. Additional datapoints may be chosen from a ranked list of datapoints in the second dataset. Ranking may be based on empirically defined bias.
  • the protected attributes estimation model may provides a confidence metric associated with the predicted protected attribute.
  • An augmented dataset may be generated based on augmenting the second dataset with predicted protected attributes from the protected attributes estimation model.
  • Embodiments disclosed include a method comprising obtaining, an augmented dataset, the augmented dataset comprising datapoints with protected attributes predicted from an estimation model; training a plurality of bias mitigation models based on subsets of the augmented dataset; obtaining a fairness metric and an accuracy metric for each of the plurality of the bias mitigation models; and choosing one model from the plurality of bias mitigation models based on at least one of the fairness metric and the accuracy metric.
  • the subsets of the augmented dataset may be generated based on statistical information of the protected attributes of the augmented dataset.
  • the statistical information may be generated by 3 KILPATRICK TOWNSEND 780676821 the estimation model.
  • the statistical information may include a confidence metric associated with a predicted protected attributed.
  • Embodiments disclosed include a method with a first stage and a second stage.
  • the first stage may comprise obtaining, a protected attributes (PA) known dataset and a PA unknown dataset; training a bias estimation model based on datapoints from the PA known dataset; generating, estimates for protected attributes based on the trained bias estimation model; augmenting the PA unknown dataset to generate an augmented dataset.
  • PA protected attributes
  • the second stage may comprise obtaining the augmented dataset; generating a plurality of subsets of the augmented dataset; training a plurality of bias mitigation models based on the plurality of subsets of the augmented dataset; and choosing one of the trained plurality of bias mitigation models as a final mitigation model.
  • Additional datapoints may be added to the PA known dataset through an active learning process.
  • the active learning process may be performed by annotating datapoints in the PA unknown dataset with protected attributes.
  • a ranked entropy model may be used to select additional datapoints during training in the second stage.
  • Embodiments may include a system comprising one or more processor coupled to a computer readable medium, the computer readable medium comprising instructions, that when executed by the one or more processors perform any of the preceding methods.
  • FIG.1A illustrates an example dataset with protected attributes.
  • FIG.1B illustrates a two stage framework to obtain fairness in machine learning applications.
  • FIG.2 illustrates an example framework to generate a protected attributes estimation model.
  • FIG.3 illustrates an example framework to generate and select one of a plurality of fair models.
  • FIG.4 illustrates an example method related to a protected attributes estimation model. 4 KILPATRICK TOWNSEND 780676821 [0017]
  • FIG.5 illustrates an example method related to generating fair models.
  • FIGS.6 , 7, and 8 illustrate performance and fairness aspects related to machine learning models.
  • FIG.9 shows an example computing system according to an embodiment of the present disclosure.
  • a “server computer” may include a powerful computer or cluster of computers.
  • the server computer can include a large mainframe, a minicomputer cluster, or a group of servers functioning as a unit.
  • a server computer can include a database server coupled to a web server.
  • the server computer may comprise one or more computational apparatuses and may use any of a variety of computing structures, arrangements, and compilations for servicing the requests for one or more “client computers.”
  • a “memory” may include any suitable device or devices that may store electronic data.
  • a suitable memory may comprise a non-transitory computer readable medium that stores instructions that can be executed by a processor to implement a desired method. Examples of memories include one or more memory chips, disk drives, etc. Such memories may operate using any suitable electrical, optical, and/or magnetic mode of operation.
  • a “processor” may include any suitable data computation device or devices.
  • a processor may comprise one or more microprocessors working together to accomplish a desired function.
  • the processor may include a CPU that comprises at least one high-speed data processor adequate to execute program components for executing user and/or system generated requests.
  • the CPU may be a microprocessor such as AMD’s Athlon, Duron and/or Opteron; IBM and/or Motorola’s PowerPC; IBM’s and Sony’s Cell processor; Intel’s Celeron, Itanium, Pentium, Xenon, and/or Xscale; and/or the like processor(s).
  • a “uniform distribution” may refer to a probability distribution (e.g., probability density function or probability mass function) where possible values associated with a random variable are equally possible.
  • a fair dice is an example of a system corresponding to a uniform distribution (in that the probability of any two rolls are equal).
  • non- uniform distribution may refer to a probability distribution where all possible values or 5 KILPATRICK TOWNSEND 780676821 intervals are not equally possible.
  • a Gaussian distribution is an example of a non-uniform distribution.
  • “Classification” may refer to a process by which something (such as a data value, feature vector, etc.) is associated with a particular class of things. For example, an image can be classified as being an image of a dog.
  • “Anomaly detection” can refer to a classification process by which something is classified as being normal or an anomaly.
  • An “anomaly” may refer to something that is unusual, infrequently observed, or undesirable.
  • a spam email may be considered an anomaly, while a non- spam email may be considered normal.
  • Classification and anomaly detection can be carried out using a machine learning model.
  • the term “artificial intelligence model” or “machine learning model” can include a model that may be used to predict outcomes to achieve a pre-defined goal.
  • a machine learning model may be developed using a learning process, in which training data is classified based on known or inferred patterns.
  • Machine learning can include an artificial intelligence process in which software applications may be trained to make accurate predictions through learning. The predictions can be generated by applying input data to a predictive model formed from performing statistical analyses on aggregated data.
  • a model can be trained using training data, such that the model may be used to make accurate predictions.
  • the prediction can be, for example, a classification of an image (e.g., identifying images of cats on the Internet) or as another example, a recommendation (e.g., a movie that a user may like or a restaurant that a consumer might enjoy).
  • a “machine learning model” may include an application of artificial intelligence that provides systems with the ability to automatically learn and improve from experience without explicitly being programmed.
  • a machine learning model may include a set of software routines and parameters that can predict an output of a process (e.g., identification of an attacker of a computer network, authentication of a computer, a suitable recommendation based on a user search query, etc.) based on feature vectors or other input data.
  • a structure of the software routines e.g., number of subroutines and the relation between them
  • the values of the parameters can be determined in a training process, which can use actual results of the process that is being modeled, e.g., the identification of 6 KILPATRICK TOWNSEND 780676821 different classes of input data.
  • machine learning models include support vector machines (SVM), models that classify data by establishing a gap or boundary between inputs of different classifications, as well as neural networks, collections of artificial “neurons” that perform functions by activating in response to inputs.
  • the model may include linear regression, logistic regression, convolutional neural network (CNN), deep recurrent neural network (e.g., fully-connected recurrent neural network (RNN), Gated Recurrent Unit (GRU), long short-term memory, (LSTM)), transformed-based methods (e.g.
  • a machine learning model can be trained using “training data” (e.g., to identify patterns in the training data) and then apply this training when it is used for its intended purpose.
  • a machine learning model can be trained using “training data” (e.g., to identify patterns in the training data) and then apply this training when it is used for its intended purpose.
  • a machine learning model may be defined by “model parameters,” which can comprise numerical values that define how the machine learning model performs its function. Training a machine learning model can comprise an iterative process used to determine a set of model parameters that achieve the best performance for the model.
  • Supervised learning models can be trained in various ways using various cost/loss functions that define the error from the known label (e.g., least squares and absolute difference from known classification) and various optimization techniques, e.g., using backpropagation, steepest descent, conjugate gradient, and Newton and quasi-Newton techniques.
  • DETAILED DESCRIPTION Embodiments include methods and systems for correcting algorithmic bias in machine learning or artificial intelligence frameworks.
  • the method may comprise any combination of the following features.
  • the method may comprise obtaining a first dataset, wherein the first dataset contains features or information related to at least one protected attribute.
  • the first dataset may be a limited dataset for training purposes and smaller than a second dataset.
  • a second 7 KILPATRICK TOWNSEND 780676821 dataset may be obtained, wherein the second dataset does not contain classifications, information, or tags related to protected attributes.
  • a protected attributes estimation model can be trained using first dataset. At least one protected attribute can be estimated for data within the second dataset.
  • This disclosed technology includes an upstream and downstream-based framework that leverages active learning to predict protected attributes from limited labeled data and utilizes these predictions into the learning process to achieve fairness in the target task.
  • the upstream model employs active learning for accurate protected attribute prediction using limited labeled data, while the downstream model incrementally samples low-biased training examples for fair learning based on the upstream estimates.
  • the framework can comprise two stages.
  • the first stage also referred to as the upstream stage or PA estimation model generation stage
  • the upstream stage can utilize active learning to generate a protected attributes estimating model.
  • the first stage or upstream stage can comprise any of the steps described with respect to FIG.2.
  • the second stage can comprise using a selected PA estimating model to generate a number of fair models.
  • the second stage can be considered to be a downstream stage.
  • the second stage or downstream stage can comprise any of the steps described with respect to FIG.3.
  • Fairness generally refers to correcting bias in decision processes (e.g., outputs) implemented using machine learning models. Decisions made by computers after a machine-learning process may be considered unfair if they were based on variables (also referred to as features or attributes) considered sensitive, such as for example, race, gender, sexuality, religion.
  • Fairness may be viewed from a group perspective (e.g., a specific group is not discriminated against) and from an individual perspective (e.g., whether similar individuals are treated similarly when excluding a sensitive variable).
  • ML algorithms may affect people in unfair ways when used to automate decision making, such as in legal, academic, or other constext.
  • a machine learning process may become unfair (also referred to as being biased) due to various reasons.
  • fairness may be defined in terms of a proportion of a class that receives a particular outcome.
  • the threshold may be a defined percentage of the other group.
  • the threshold may be a fixed percentage (e.g., the proportion of green toy should be at least 80% of the proportion of blue toys for a particular classification).
  • the particular portion may be based on legislation or other rules. For example, Title VII of the Civil Rights Act of 1964 provides that there is disparate impact when the ratio of probabilities of a particular outcome between two groups is less than 80%.
  • FIG.1 illustrates an example dataset 100.
  • Dataset 100 is a fabricated dataset for illustrative purposes and describes features related to various toys. It is to be understood that dataset 100 may contain any number of rows (e.g., datapoints) and columns (e.g., each column related to a feature). Dataset 100 describes various features related of toys, identified with IDs 1 to 10 in column 110.
  • the toy may have a name (Toy_Name in column 111), color (column 112), shape (column 113), manufacturer (column 114), material type (Material_Type in column 115), age group (Age_Group in column 116), price (column 117), country of origin (column 118), supply chain method (column 119), and demand (column 120).
  • Demand which is either low, medium, or high, may be a target task or output on which the dataset can be trained.
  • a machine learning model may classify new toys into one of the three demand categories. Additionally, the machine learning model may provide statistical information related to an output, such as a confidence metric related to the output.
  • the classification may be the demand for a toy as low, medium, or high. In other examples, the classification may be binary, such as a yes or a no output from the model.
  • Specific features within dataset 100 may be considered as protected features.
  • Protected features may also be referred to herein as protected attributes (PAs).
  • PAs protected attributes
  • the country of origin for the toy may be deemed a protected attribute.
  • this protective attribute should have a minimal or no predictive effect on the trained machine learning model.
  • the supply chain method chosen for the toy may be a protected attribute.
  • the supply chain method may be thought to be unrelated to how strong the demand for a particular toy would be.
  • FIG.1 also illustrates dataset 199.
  • dataset 199 is derived from dataset 100 by removing elements from the dataset.
  • a protected attribute is accessible for only three points of data, corresponding to IDs 1, 5, and 6.
  • the training data containing protected attributes is limited, it would be difficult to robustly train a machine learning model based on dataset 199 while also ensuring that the predictive effect of the 10 KILPATRICK TOWNSEND 780676821 protected attributes is minimized.
  • the fairness of models trained on the example datasets 100 and 199 may be examined.
  • Various approaches may be taken with respect to ensuring that a trained model is fair.
  • a fairness metric may be measured for an algorithm during the training of the algorithm. If the fairness metric is violated, an objective function may be penalized to better reflect the fairness criteria described.
  • a constraint may be placed on the training of the object function that it can respect the fairness criteria. This may also be referred to as constraint-based optimization approach.
  • constraint-based optimization approach relies on the ability to distinguish between group membership and requires group membership information for all the data instances in the dataset.
  • FIG.1B illustrates framework 150 with an upstream stage 151 (also referred to as a first stage or a protected attributes (PA) estimation stage) and a downstream stage 152 (also referred to as a second stage or a fair model generation stage).
  • upstream stage 151 also referred to as a first stage or a protected attributes (PA) estimation stage
  • downstream stage 152 also referred to as a second stage or a fair model generation stage
  • a dataset D may be used to train a bias estimation model (assigned symbol Mbe and also referred to as a PA estimation model).
  • Dataset D may comprise of two subsets – a first subset of data containing protected attributes information (assigned symbol D pal and also referred to as PA known dataset) and a second subset which does not contain information related to protected attributes (assigned symbol D pau and also referred to as PA unknown dataset).
  • the training of the bias estimation model is further discussed herein.
  • the bias estimation model may predict protected attributes for datapoints within the PA unknown dataset (Dpau) and also provide quantitative information about those predictions (e.g., the confidence in the prediction expressed as a probability).
  • Information outputted by the bias estimation model may be appended to the dataset without protected attributes and used in the downstream stage 152. Once appended, this may be referred to as the augmented dataset (assigned symbol D pauPL ).
  • the augmented dataset may also optionally contain statistic information related to the predictions obtained from the bias estimation model.
  • multiple models may be trained using the augmented dataset (D pauPL ). Subsets of the augmented dataset may be created through a selection process.
  • Each such subset of the augmented dataset maybe used to train a different bias mitigation model.
  • the selection process may be based on, for example, a confidence range of each prediction of the bias estimation model.
  • “i” subsets may be generated and “i” bias mitigation models may be generated. These “i” bias mitigation models may be referred to as Mbmi.
  • One model from the generated “i” bias mitigation models may be chosen based on performance criteria (e.g., for use in a production environment. This model may be assigned symbol M bm .
  • the two stage model may be advantageous as it allows for bias correction (e.g., through training a bias mitigation model) even when the underlying protected attributes are not known.
  • the two stage model thus allows for training of fair models (e.g., bias mitigation models) on a small dataset PA known while retaining robust performance. Further, the two stage model may not need information from any the PA known dataset (Dpal) to train the bias estimation model.
  • PAs protected attributes estimation model may be trained on a subset of data which has protected attributes.
  • the protected attributes estimation model may also be referred to as an upstream model or a bias estimation model.
  • the protected attributes estimation model may take as an input data which does not contain the protected attributes (e.g., the PA unknown dataset D pau ) and output the protected attributes for that data.
  • the training of the PA estimation model may be done through active learning. Active learning is semi-supervised learning where the performance of the model may be gradually improved. A small subset of the training data originally be used for training with additional and more valuable data from the remaining of the dataset being used to supplement the training of the model. During the training of the PA estimation model, an attempt will be made to minimize the number of queries to the protected attributes. If an iteration of the trained protected attributes model is not robust or does not meet certain performance criteria, additional datasets may be requested.
  • FIG.2 illustrates the framework 200 which may be used to generate a PA estimation model.
  • Figure 2 illustrates a datasets 210A, 210B, and 210C, and PA estimation models 220A, 220B, and 220C corresponding to and trained on each dataset respectively.
  • An active learning step between the data sets is also illustrated.
  • the data in dataset 210A may be supplemented to create dataset 210B through an active learning process.
  • dataset 210A or 210B may be supplemented to create dataset 210C.
  • Active learning allows an algorithm to determine whether to obtain additional information regarding a dataset based on a cost-benefit analysis.
  • an algorithm may proactively select a subset of examples to be labeled next from the pool of unlabeled data.
  • Active learning may include stream-based selective sampling techniques, pool-based sampling techniques, and membership query synthesis techniques. In stream- based selective sampling, a decision is made for data instances and whether it would be 13 KILPATRICK TOWNSEND 780676821 worthwhile to query that data instance for a label.
  • Datasets 210A, 210B, and 210C may be subsets of a larger data set which include information about protected attributes. As one example, with reference to FIG.1, rows 1, 5, and 6 of dataset 199 may form dataset 210A.
  • certain information regarding one or more protected attributes may be missing.
  • this information may be supplemented through an active learning technique.
  • a human operator may provide information regarding the supply chain method in row 1 and the country of origin in row 5.
  • This information may be included in dataset 210B, on which training may be performed to generate PA estimation model 220B, which may be more accurate than PA estimation model 220A as more data is available.
  • Additional information may transform dataset 210A or dataset 210B into dataset 230C by including additional selected examples for which protected attributes are added.
  • PA estimation models 220A, 220B, and 220C may be machine learning models.
  • PA estimation model may be a machine learning model that is trained to generate predicted classifications for a value of a protected attribute. These output classifications may be one or more protected attributes.
  • the PA estimation models 220A, 220B, and 220C may generate additional statistical information along with the generated classifications.
  • metrics related to the confidence of the classification may be provided by PA estimation models.
  • additional columns may be appended to the table illustrated with respect to FIG.1 to include statistical information.
  • 14 KILPATRICK TOWNSEND 780676821 Statistical performance for a PA estimation model may be generated upon training of the model. This statistical information may include a probability that a particular data point contains a protected attribute. For example, the protected attribute estimation model may be tested on the validation dataset and rank the confidence of the model on these data instances and find out at which datapoint(s) the model is not confident. Among the least confident data instances, a subset of the data (e.g., the last ten data instances) may be selected to be annotated with through an active learning framework.
  • ⁇ Mbe is a machine learning model for bias estimation. Mbe may be trained on the PA known dataset (Dpal). After training, Mbe may generate a protected attribute for each data instance in the PA unknown dataset. M be may be trained in upstream stage. Mbe may also be referred to as a PA estimation model.
  • D pal A subset of dataset D in which protected attributes are known (e.g., labeled). This is also referred to as the PAs known dataset. This dataset may be updated with additional datapoints for which PAs are known through active learning.
  • ⁇ Ppau An estimated probability outputted for one or more points from a PA estimation model.
  • ⁇ D pau A subset of dataset D in which protected attributes are unknown (e.g., unlabeled). This is also referred to as the PAs unknown dataset.
  • ⁇ D A dataset which contains data with both known protected attributes and unknown protected attributes. It may also be referred to as a complete dataset.
  • table 199 may be an example of such a dataset as it contains both datapoints for which protected attributes are known and datapoints for which 15 KILPATRICK TOWNSEND 780676821 protected attributes are unknown.
  • D is a dataset formed from two subsets Dpal and Dpau. In some examples, the size of Dpau>> Dpal.
  • ⁇ D paupl Dataset generated from D pau using M be , and includes estimate protected attributes. This is also referred to as the augmented dataset.
  • the PA estimation model ( ⁇ ⁇ ⁇ ) may be trained with the following method.
  • ⁇ ⁇ ⁇ is an ML model which takes data ⁇ and produces outcome ⁇ ⁇ for each individual with a particular protected attribute ⁇ ⁇ .
  • the selection of samples from the set of ⁇ ⁇ ⁇ ⁇ may include the use of a ranked query approach. Examples of such an approach includes the batch-mode active learning and ranked batch-mode active learning techniques described in Cardoso, T. N. C., Silva, R. M., Canuto, S., Moro, M. M., & Gonçalves, M. A. “Ranked batch-mode active learning.” Information Sciences, vol.379, no.10, I.313-337, 2017.
  • Q u may be calculated for each datapoints, which may then be ranked based on the calculated Qu values.
  • protected attributes for unlabeled data may be generated using the PA estimation model.
  • Statistical information may also be output by Mbe.
  • the data instances in ⁇ ⁇ ⁇ ⁇ may be analyzed to generate the dataset ⁇ ⁇ ⁇ ⁇ pl.
  • Each data instance in ⁇ ⁇ ⁇ ⁇ pl may contain statistical information (e.g., uncertainty information, probability information, confidence values).
  • the set ⁇ ⁇ ⁇ ⁇ pl may be sampled to produce a number of models in the second stage. V.
  • stage two also referred to as a downstream stage, second stage, or a fair model training stage
  • multiple machine learning models may be trained.
  • the training of each of the multiple machine learning models may be based on a different subset of the augmented dataset (D ⁇ ⁇ ⁇ ⁇ ⁇ ).
  • the augmented dataset includes predicted PAs, outputted from the bias estimation model discussed above, which are augmented to the PA unknown dataset (D ⁇ ⁇ ⁇ ).
  • Each of the machine learning models (which may be referred to as fair models and assigned symbol Mbmi, where i is an index) may be measured for accuracy and fairness.
  • One of the multiple fair models may be selected for use in predicting on a target task.
  • the final selected fair model may be represented as M bm .
  • the selection of the final model may be based on requirements from the final model.
  • the training of a fair model may be performed on the augmented dataset ( ⁇ ⁇ ⁇ ⁇ pl), rather than the PA dataset (Dpal) or complete dataset (D), as training on either of those datasets would propagate any bias that is present in the data into the fair model.
  • the propagation of bias may be mitigated by using the augmented dataset where statistical information about each predicted PA is known. 17 KILPATRICK TOWNSEND 780676821 A. Overview
  • FIG.3 illustrates a framework 300 to generate a bias mitigation model from dataset D.
  • the bias mitigation model may be generated from the augmented dataset (D paupl ) 320 Illustrated in FIG.3 is PA Estimation Model 310, PA unknown dataset (D pau ) 311, augmented dataset 320, datasets generated from uncertainty sampling 330A, 330B, and 330C, a plurality of bias mitigation models 340A, 340B, and 340C, and a final bias mitigation model 350.
  • PA estimation model 310 may be the bias estimation model (Mbe) described in FIG.2.
  • Dataset 320 may be the augmented dataset ( ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ), which may include information outputted from the PA estimation model 310 and the PA unknown dataset 311.
  • Datasets 330A, 330B, and 330C may be chosen based on statistical information (e.g., uncertainty of predictions datapoints within the datasets) from dataset 320.
  • Datasets 330A, 330B, and 330C may be subsets of dataset 320.
  • Bias mitigation models 340A, 340B, and 340C may be models which are trained on datasets 340A, 340B, and 340C respectively. Although only 3 datasets and 3 corresponding models are illustrated, any “i” number of datasets may be selected to train “i” potential models.
  • Final bias mitigation model 350 may be chosen from one or more models based on criteria, such as for example performance, accuracy and bias in the model.
  • each bias mitigation model (340A, 340B, 340C, etc.) may be based on a modification to the training data rather than a modification to the underlying machine learning model.
  • Each datapoint in the augmented dataset 320 contains both (i) features unrelated to the (e.g., features which are not protected attributes) and (ii) protected attributes 18 KILPATRICK TOWNSEND 780676821 that are estimated from the bias estimation model (Mbe).
  • Each estimated protected attribute outputted from the bias estimation model may be associated with statistical information.
  • the statistical information may be the confidence of the value estimated for the protected attribute by the bias estimation model.
  • Datasets 330A, 330B, and 330C may be generated from the augmented dataset 320 ( ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ). Each dataset may be based on statistical attributes of the protected attributes, such as how confident the bias estimation model may be regarding a prediction.
  • Dataset 330A may contain protected attributes for which it is harder to distinguish between group membership as the confidence about a prediction is low.
  • Dataset 330B may include additional datapoints for which there is a higher confidence regarding their prediction.
  • Dataset 330C may include even additional datapoints for which there is an even higher confidence regarding their prediction.
  • each dataset may be used to train a different bias mitigation model.
  • the accuracy of the trained bias mitigation model may increased.
  • fairness may decrease as points which are more biased are included.
  • the confidence may be represented by a value from a range, such as between 0 and 1.
  • For binary values may be between a range (e.g., 0 to 1) such that a value of 0 implies a 100% confidence that the estimated value is the first of the binary values (e.g., round) while a value of 1 may imply a 100% confidence that the estimate value is the second of the binary values (e.g., square).
  • a perfect prediction such as between 0 and 1.
  • a confidence metric of .5 may imply that the bias estimation model is 0% confident about the value it has predicted (e.g., the bias estimation model is entirely unsure whether the protected attribute should be round or square). Confidence metrics between 0 and .5, and .5 and 1 may similarly denote confidences which are not extremes. For example, a confidence metric of .25 may imply that there is a 75% confidence that predicted value is the first of the binary values (e.g., round). [0071] Subsets of the augmented dataset 320 (D paupl ) may be determined based on values which lay within a range of confidence metrics. As discussed above, each datapoint in the augmented dataset 320 will have a confidence metric associated for the protected attribute predicted for that datapoint.
  • the datapoints within the augmented dataset may be divided into subsets. For instance, datasets that are iteratively larger from a central value (e.g., .5) may be used for training bias mitigation models.
  • a k th subset may include all points which have confidence metrics between [.5-kq, .5+kq], where q is an arbitrary real number and k is an index. As k increases, each subset may be nested within a next subset. Each subset may be used to train a bias mitigation model.
  • the m th subset will contain all datapoints from the n th subset and additional datapoints. However, the m th subset will contain a broader range of confidence metrics which are further from .5 (corresponding to a confidence of 0%). As a 0% confidence implies a completely “neutral” value for a protected attribute, values farther from 0% may reflect bias for that predicted protected attributes. Thus, including such values will reduce fairness in a bias mitigation model generated. However, for robust training, additional datapoints are required to improve precision. Thus, each bias mitigation model trained on a different dataset may contain different characteristics in terms of tradeoffs between fairness and precision.
  • the subsets may be chosen by increasing the number of datapoints within a confidence range but not increasing the confidence range. Additionally, pre-processing steps may be used to ensure that a robust sample of datapoints is included in the subsets.
  • C. Training Bias Mitigation Model from Datasets [0074] Datasets 330A, 330B, and 330C may be created from the augmented dataset 320 ( ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ). For example, each dataset may be based on intervals related to statistical information of the generated protected attributes. For instance, dataset 330A may include all datapoints in ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ , which have a confidence metric of .45–.55 associated with an estimated protected attribute.
  • Dataset 330B may include a larger range.
  • dataset 330B may include all datapoints in ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ which have a confidence metric of .4–.6 associated with them.
  • Dataset 330C may contain an ever larger range.
  • dataset 330C may have contain all data points between .3 and .7 in their confidence metric.
  • a pre- processing algorithm may be used to process the data instances being selected for datasets 330A, 330B, and 330C.
  • fairness preprocessing algorithm such as the disparate impact remover may be used.
  • a disparate impact remover is a preprocessing technique that edits feature values to increase group fairness while preserving rank-ordering within groups.
  • DIR techniques can reduce or remove inherent biases from a set of data to minimize the effect of such biases on an algorithm trained with that data.
  • Certain DIR algorithms employ an optimization approach to transform a dataset with the aim of eliminating bias while preserving data utility to the extent possible.
  • Certain DIR methodologies are rooted in the legal concept of disparate impact, making them relevant for real-world applications when legal compliance is critical.
  • the same DIR technique applied to training data may also be applied to production data (e.g., data which is to be classified by a machine learning model).
  • Datasets may be chosen in this manner to obtain robust models.
  • a confidence metric e.g., probability
  • a value of 1 ⁇ 2 in this example may imply perfectly unbiased datapoint as the model is unable to determine which value to assign for the protected attribute for this point.
  • a final model 350 may be selected from the multiple trained models based on accuracy and fairness requirements. Any other metric may be chosen as a basis for selecting a final model 350. Final model 350 may be selected for use in a production environment to determine outputs for data which requires classification. [0079] An additional description of training a fair model is described herein.
  • KILPATRICK TOWNSEND 780676821 ⁇ L, r, and ⁇ are the hyper-parameters for sample selection. l and r are used to gradually increase the sample sizes in each iteration in terms of the confidence score of each iteration of the PA estimation model. ⁇ .or ⁇ ⁇ is a regularizer. To refine the quality of the samples with increasing iterations, we regularize by reducing the number of samples with the regularization factor ⁇ . ⁇ ⁇ ⁇ ⁇ : This is a dataset which trains in the second stage (downstream stage). It may also be referred to as a bias mitigation dataset as it is intended to mitigate bais.
  • ⁇ ⁇ ⁇ is a regularizer.
  • represents the distance from the current lower biased region to the current high biased region. Stated alternatively, this represents a range of values.
  • ⁇ is the number of examples in the new examples pool
  • the threshold for what is included in a training data set can be increased based on, for example, the parameters above. For example, in binary cases, threshold parameters may be entered around .5.
  • the training dataset may be constructed by including additional instances of data with each iteration.
  • the fair model (a chosen final Mbm) may be chosen and put into a production environment.
  • Example methods are illustrated below related data selection, active learning, and training of machine learning models (e.g., protected attributes estimation models (in a first stage), classification models (in the second stage)) according to embodiments of the disclosed technology.
  • A. Generating Protected Attribute Estimation Model [0082] With respect to FIG.4, an example method 400 is provided to generate an upstream estimation model.
  • the upstream estimation model may also be referred to as a protected attributes estimation model.
  • Example method 400 may utilize or combine any of the techniques described with respect to the disclosure herein.
  • a first dataset can be obtained.
  • the first dataset can contain features related to at least one protected attribute.
  • the first dataset can be selected or generated from a larger dataset which contains protected attributes.
  • first dataset may be PA known dataset (Dpal) which may be obtained from dataset D, which are described above.
  • a second dataset can be obtained.
  • the second dataset may not contain features, information, tags, or classifications related to at least one protected attribute.
  • the second dataset can be larger than the first dataset and representative of data which would be classified as part of a target task.
  • second dataset may be the PA unknown dataset (Dpau,)which may be obtained from dataset D, which are described above.
  • a first dataset can be used to train a protected attributes estimation model.
  • the protected attributes estimation model can be a machine learning model which can estimate information or classifications related to at least one protected attribute.
  • the protected attributes estimation model may be trained for a single attribute or multiple attributes.
  • the protected attributes estimation model can be updated.
  • a human expert can annotate or provide information on a selected number of examples of the second dataset to allow for a next iteration or version of the protected attributes estimation model.
  • Step 408 may be iterated or repeated as required.
  • the annotation can be simulated.
  • additional labels can be revealed to simulate interactive annotation.
  • a subset of the dataset which is labeled may be used to train a PA estimation model (e.g., 30% of the labeled data).
  • the remainder of the dataset may be treated as “unlabeled” despite the PA for that data being known.
  • the PA estimation model may be used to make predictions regarding the protected attributes for the other data (e.g., the other 70% of the labeled data). For those predictions, the datapoints where uncertainty is the highest may be chosen.
  • those datapoints may be provided to further train the PA estimation model by adding those specific instances to the training dataset.
  • datapoints from the second dataset e.g., PA unknown dataset
  • the updated first dataset may be used in the training of iteration.
  • the datapoints chosen from the second dataset may be used to be included in the first dataset.
  • the estimation can be based on the trained protected attributes estimation model.
  • the estimation (also referred to as a prediction) by the protected attributes estimation model may also provide statistical information related to the prediction.
  • method 400 can be considered an upstream estimation model and be used in conjunction with a downstream model.
  • updating of the protected attributes estimation model can occur iteratively, based on some criteria, such as the resource, budget, number of examples, or availability of human experts.
  • the method can further comprise ranking the selected number of examples from the second dataset by empirically defined bias.
  • One output of method 400 may be a protected attributes model. This model may be used on a dataset, such as dataset Dpau, which does not contain any information related to protected attributes. B.
  • FIG.5 illustrates method 500.
  • Example method 500 can be used to generate a number of fair models and select one of the fair models as a final model based on fairness, bias, accuracy, or other metrics.
  • Example method 500 may utilize or combine any of the techniques described with respect to the disclosure herein inclusive of the appendices attached.
  • the generated fair models may be considered downstream as they are based on sampling predicted protected attributes from an unlabeled dataset and generated “downstream” from such unlabeled data.
  • the fairness models may be referred to as bias mitigation models (Mbm or Mbmi).
  • a dataset containing protected attributes predicted using a PA estimation model may be obtained (e.g., an augmented dataset).
  • a dataset containing predicted protected attributes e.g., an augmented dataset (Dpaupl)
  • D pau PA unknown dataset
  • the dataset D paupl may also contain statistical information about each predicted PA for each data instance. This information may be used as described below to create training sets.
  • the statistical information may be obtained from the PA estimation model.
  • 25 KILPATRICK TOWNSEND 780676821 [0093]
  • statistical data from the dataset may be sampled to obtain a set of training data.
  • the sampling may be performed based on a range associated with a statistical metric of the predicted protected attribute (e.g., confidence of a prediction expressed as a probability). For example, a small range may be chosen at this step (such as 48-52% confidence).
  • a preprocessing algorithms may be applied on the set of training data. These may include fairness preprocessing algorithm such as a disparate impact remover.
  • a bias mitigation model may be trained using the set of training data, as modified by the preprocessing algorithm. Statistical information and performance information of the model may also be obtained at this step.
  • a fairness metric and an accuracy metric can be obtained for one or more of the trained models.
  • An accuracy metric may be one such as precision of the predictions made for a target task (e.g., a target classification).
  • a fairness metric may include one or more measures of fairness, including measures based on group fairness, group unawareness, demographic parity, disparate impact, equal opportunity, equal odds, positive predicted value parity, false positive rate parity, and negative predicted value parity.
  • a confusion matrix may be used to study one or more fairness metrics. The confusion matrix may also be used to determine an accuracy metric.
  • An accuracy metric may correspond to the number of correct predictions/ total number of predictions.
  • a confusion matrix may include a matrix representing, for a dataset, the number of true positives (the total number of outcomes where the model correctly predicts the positive class), true negatives (the total number of outcomes where the model correctly predicts the negative class), false positives (the total number of outcomes where the model incorrectly predicts the positive class), and false negatives (the total number of outcomes where the model incorrectly predicts the negative class.)
  • the various numbers in the matrix may be compared or evaluated to determine a fairness metric and an accuracy metric.
  • Demographic parity may refer to the extent a protected attribute affects an outcome. For example, for a binary group, a prediction should be statistically independent of the protected attribute.
  • Various metrics may be calculated based on demographic parity.
  • KILPATRICK TOWNSEND 780676821 metric may be calculated as the difference between such probabilities or ratio between such probabilities. For a perfectly fair model, the accuracy ratio would be expected to be “1.”
  • Disparate impact may refer to when one group has a higher probability of getting a certain outcome than another outcome.
  • a fairness metric may be a binary function which returns a value of 0 or 1 when the ratio between the outcome for the two groups is higher than a certain ratio.
  • Equal opportunity may refer to the true positive rate between two groups.
  • a fairness metric may look at the difference between the true positive rate between two (or more) groups. For example, the true positive may be the payback of a loan.
  • Other variations based on an equal opportunity may also form the basis for a fairness metric. For example, a false positive rate may be measured between two group and metrics may be based on the same.
  • Positive predicted value parity and negative predicted value parity can refer to when two groups have the same positive predicted value or the same negative predicted value.
  • the positive predicted value may be defined in a binary case (e.g., the likelihood that an event occurs).
  • a fairness metric may be based on variations from the parity values expected in both cases.
  • the described example fairness metrics and methods of calculations may further be modified based on requirements.
  • additional models can be generated by iterating steps 504, 506, and 508. The iteration may occur by expanding the statistical range to obtain additional points of data. Thus, additional sets of training data may be obtained to train additional models.
  • Step 509 may be undertaken as a sufficient number of examples may not be available based on the selected number of examples or datapoints, additional examples or datapoints can be obtained.
  • additional datapoints can be selected based on criteria described herein, such as using a ranked entropy model.
  • entropy ⁇ u can be 27 KILPATRICK TOWNSEND 780676821 ranked for the i th iteration of the set of training data ⁇ ⁇ ⁇ .
  • ⁇ ⁇ ⁇ may be a dataset with both non-estimated data attributes and protected attributes estimated from a PA estimatnion model.
  • Examples within ⁇ ⁇ ⁇ ⁇ may be ranked from high entropy examples to low entropy examples in order to prune that dataset to select the most informative and difficult-to-predict examples from the target task’s perspective. Further, to account for the quality of the examples from the PA estimation models example, we may select a number, ⁇ ⁇ ⁇ from ⁇ ⁇ ⁇ ⁇ . In some examples, the following equation may be used to determine the number: ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ ( ⁇ exp ⁇ ⁇ ). ⁇ and ⁇ have been previously defined above. ⁇ may be chosen to ensure a minimum number of samples are selected.
  • the output of a protected attributes estimation model such as a predicted feature or group (e.g., race or gender) can be provided to a fairness pre-processing algorithm, such as a disparate impact remover algorithm, to modify feature values to improve group or feature values.
  • a fairness pre-processing algorithm such as a disparate impact remover algorithm
  • the training process may be iterated to generate additional fair models.
  • the statistical range may be expanded to include additional data points for training. For example, as explained above with respect to FIG.3, the confidence metric associated with protected attributes may be expanded.
  • the iterative process of training fair models may continue until a sufficient number of models are generated. In some examples, the iterative process may continue until a model with sufficient performance is determined.
  • one of the selected final models can be utilized in a production or other environment.
  • the selected model can reduce algorithmic bias or be selected based on threshold criteria for bias or fairness.
  • the selection of a final model can be used in a production environment to analyze data while minimizing bias.
  • the selection of models can be determined based on, for example, a tradeoff between accuracy and expected fairness of the model. In some examples, the model can be based on the expected target database or use case.
  • the first dataset is the Adult Income dataset.
  • the dataset is derived from US census data.
  • a target task for training a machine learning model is a binary classification problem to predict if an individual’s income exceeds $50,000 a year.
  • the dataset contains features such as gender, race, marital status, capital gains, etc.
  • gender was chosen as a protected attribute while predicting the income level.
  • the task a binary classification problem, that is whether a user will default on their credit card payment or not.
  • the dataset includes 23 attributes such as age, payment history, bill amount, and previous payment amount. In this dataset, age was considered to be a protected attribute.
  • FIG.6 illustrates example results on both the adult income dataset and the DCC dataset. Illustrated in FIG.6 is table 610. Table 610 illustrates the results obtained from the two datasets (the Adult Income Dataset and the DCCC dataset) with and without proxies included for a baseline model, a disparate income removal model, a reweighting model, and models generated using the methods described herein (labeled disclosed with ⁇ ⁇ ⁇ and disclosed without ⁇ ⁇ ⁇ ).
  • Table 610 illustrates a correlation between proxy features and target tasks classifications.
  • removing proxy features may be beneficial in enhancing model fairness.
  • a trend in Table 610 is the improvement in terms of disparate impact (DI) fairness metric from the baseline to (i) a disparate impact remover (DIR) and reweighting technique and (ii) to the disclosed method.
  • DI disparate impact
  • DIR disparate impact remover
  • RF XGB
  • the disclosed technology may lead to improvement in fairness for both the “w. proxy” and “w/o” proxy settings, which are further explained below. These improvements may be obtained due to the estimation of protected attributes from the first stage (upstream stage). Additionally, the DI metric obtained using DIR with the disclosed technology is better than the results obtained using DIR with the original PAs. This improvement can be attributed to the disclosed technology’s ability to select a smaller number of high-quality examples for training as compared to using the entire original training dataset, indicating the potential of the disclosed technology to enhance the effectiveness of DIR.
  • B. Details of Experimental Setup [0115] Additional detail regarding the experimental setup related to table 610 is further provided herein.
  • the datasets referenced in FIG.6 were divided into a training, validation, and test dataset.
  • the results illustrated in FIG.6 are the results for a test dataset. Following the tuning of hyperparameters for both first stage and second stage models on a validation set of data, the tuned models were used and the results reported were on the results from the test set.
  • the first set referred to as “with proxies (w. proxies),” included attributes such as relationship status and marital status that could serve as proxies for the protected attributes (PAs). These proxy features were combined with regular features, which are dependent variables relevant for predicting the target downstream task.
  • the second set referred to as "without proxies (w/o proxies)," solely consisted of regular features.
  • the feature list included regular features such as age, education, capital gain, capital loss, hours-per-week, workclass, and occupation, along with proxy features such as marital status, relationship, and native country.
  • the regular features included limit on balance, payment, bill amount, and payment amount, while the proxy feature was education.
  • a “baseline” may be created against which to compare the results from the disclosed technique models.
  • a baseline may be developed without incorporating fairness considerations in the model development.
  • DIR fuel impact remover
  • reweighting may also be considered for metrics against which to compare the results.
  • the entire training data, including the original PAs may be used as input.
  • the disclosed technique in contrast, may leverages estimated PAs and may only require only a smaller subset of the training data that includes the original PAs.
  • FIG.6 also illustrates logistic regression (LR), Random Forest (RF), and XGBoost (XGB) techniques which may be used as part of the classification model used for a first stage within an active learning framework. For instance, a model may be initiated with a seed dataset (labeled ⁇ ⁇ ), which may include 0.1% of the training examples.
  • hyperparameters may be selected as follows selected as follows. For LR “balanced" class weight, ⁇ 1 & ⁇ 2 penalty, specified a maximum 1,000 number of iterations for the solvers to converge and set the inverse of regularization strength to 0.1 may be used. For RF, the number of trees may be set to 30, a maximum depth to 20, and used the Gini criterion. For XGB logistic regression as the criterion and set the maximum depth to 6 may be used.
  • DI disparate impact ratio
  • EOD equalized odds difference
  • EOP equal of opportunity
  • FIG.7 illustrates an ablation study.
  • FIG.7 illustrates the number of data instances (e.g., data points) required to train the second stage (downstream model) both with (solid line) and without (dotted line) using the exponential decay equation for Kd discussed above.
  • Graph 710 illustrates the study with the use of proxies while graph 720 illustrates the study without the use of proxies.
  • the x-axis horizontal axis
  • the y-axis vertical axis
  • FIG.7 illustrates that employing the exponential decay equation for Kd results in the selection of a smaller number of examples while attaining similar, or better results, compared to the baseline, DIR, and reweighting approaches, particularly for models such as RF and XGB.
  • FIG.8 illustrates additional charts 810, 820, and 830 which respectively illustrate improvements in three different machine learning models, namely, logistic regression, random forest, and XGBoost, as compared to a baseline measurement.
  • logistic regression logistic regression
  • random forest and XGBoost
  • one aspect of the present technology is the gathering and use of publicly available information to preserve privacy and remove disparate impact with respect to protected attributes.
  • the spirit of the invention is to promote fairness in models by removing disparate impact to individuals and groups of individuals by not relying on protected attributes, and proxies thereof, to make predictions.
  • It is the spirit of the invention 33 KILPATRICK TOWNSEND 780676821 to remove any information which causes disparate impact due to proxies (e.g., zip code) or protected attributes.
  • proxies e.g., zip code
  • the present disclosure recognizes that the use of such personal information data, in the present technology, can be used beneficially to users.
  • the personal information data can be used to predict financial instruments or products which may be of greater interest or utility to a user in accordance with their personal information data.
  • particular offers which may aligned with a user’s interest based on personal information data may be provided to the user.
  • this gathered data may include personal information data that uniquely identifies or can be used to identify a specific person.
  • personal information data can include demographic data, location-based data, online identifiers, telephone numbers, email addresses, home addresses, date of birth, or any other personal information. However, such information is not intended to identify a specific individual but be generalized and anonymized for use.
  • the present disclosure also contemplates embodiments in which users selectively block the use of, or access to, personal information data. For example, users may “opt out” of the collection of one or more pieces of personal information during the registration of services. In other examples, users may be informed that the provided information is anonymized and not specifically tied to the specific user for identification of that user. In yet other examples, information may be deleted after a specified period of time. [0131] Moreover, it is the intent of the present disclosure that personal information data should be managed and handled in a way to minimize risks of unintentional or unauthorized access or use. Risk can be minimized by limiting the collection of data and deleting data once it is no longer needed.
  • data de-identification can be used to protect a user’s privacy.
  • De- identification may be facilitated, when appropriate, by removing identifiers, controlling the amount or specificity of data stored (e.g., collecting location data at city level rather than at an address level), controlling how data is stored (e.g., aggregating data across users), and/or other methods such as differential privacy.
  • the present disclosure contemplates that those entities responsible for the collection, analysis, disclosure, transfer, storage, or other use of such personal information 34 KILPATRICK TOWNSEND 780676821 data will comply with well-established privacy policies and/or privacy practices, including all applicable laws and regulations.
  • Non-limiting examples of policies and practices which may be used include federal and state laws, such as the Health Insurance Portability and Accountability Act (HIPAA) in the United States , the California Consumer Privacy Act (CCPA) in the State of California, or the General Data Protection Regulation (GDPR) applicable in Europe.
  • HIPAA Health Insurance Portability and Accountability Act
  • CCPA California Consumer Privacy Act
  • GDPR General Data Protection Regulation
  • the process of training a machine learning model can involve determining the set of parameters that achieve the “best” performance, often based on the output of a loss or error function.
  • a loss function typically relates the expected or ideal performance of a machine learning model to its actual performance. For example, if there is a training data set comprising input data paired with corresponding labels (indicating, e.g., whether that input data corresponds to normal data or anomalous data), then a loss function or error function can relate or compare the classifications or labels produced by the machine learning model to the known labels corresponding to the training data set.
  • the loss function may take on a small value (e.g., zero), while if the machine learning model produces labels that are totally inconsistent with the labels corresponding to the training data set, then the loss function may take on a larger value (e.g., one, a value greater than one, infinity, etc.).
  • the training process can involve iteratively updating the model parameters during training rounds, batches, epochs, or other suitable divisions of the training process.
  • a computer system can evaluate the current performance of the model for a particular set of parameters using the loss or error function.
  • the computer system can use metrics such as the gradient and techniques such as stochastic gradient 35 KILPATRICK TOWNSEND 780676821 descent to update the model parameters based on the loss or error function.
  • the computer system can predict which change to the model parameters result in the fastest decrease in the loss function, then change the model parameters based on that prediction. This process can be repeated in subsequent training rounds, epochs, etc. [0137] The computer system can perform this iterative process until training is complete. In some cases, the training process involve a set number of training rounds or epochs.
  • a computer system includes a single computer apparatus, where the subsystems can be the components of the computer apparatus.
  • a computer system can include multiple computer apparatuses, each being a subsystem, with internal components.
  • a computer system can include desktop and laptop computers, tablets, mobile phones and other mobile devices.
  • FIG.9 The subsystems shown in FIG.9 are interconnected via a system bus 75. Additional subsystems such as a printer 74, keyboard 78, storage device(s) 79, monitor 76 (e.g., a display screen, such as an LED), which is coupled to display adapter 82, and others are shown. Peripherals and input/output (I/O) devices, which couple to I/O controller 71, can be connected to the computer system by any number of means known in the art such as input/output (I/O) port 77 (e.g., USB, FireWire ® ).
  • I/O input/output
  • I/O port 77 or external interface 81 can be used to connect computer system 700 to a wide area network such as the Internet, a mouse input device, or a scanner.
  • the interconnection via system bus 75 allows the central processor 406 to communicate with each subsystem and to control the execution of a plurality of instructions from system memory 72 or the storage device(s) 79 (e.g., a fixed disk, such as a hard drive, or optical disk), as well as the exchange of information between subsystems.
  • the system memory 72 and/or the storage device(s) 79 may embody a computer readable medium.
  • Another subsystem is a data collection device 85, such as a camera, microphone, accelerometer, and the like.
  • a computer system can include a plurality of the same components or subsystems, e.g., connected together by external interface 81, by an internal interface, or via removable storage devices that can be connected and removed from one component to another component.
  • computer systems, subsystem, or apparatuses can communicate over a network.
  • one computer can be considered a client and another computer a server, where each can be part of a same computer system.
  • a client and a server can each include multiple systems, subsystems, or components.
  • methods may involve various numbers of clients and/or servers, including at least 4, 20, 50, 40, 200, 500, 1,000, or 4,000 devices.
  • Methods can include various numbers of communication messages between devices, including at least 40, 200, 500, 1,000, 4,000, 50,000, 40,000, 500,00, or one million communication messages.
  • Such communications can involve at least 1 MB, 10 MB, 100 MB, 1 GB, 10 GB, or 100 GB of data.
  • Any of the computer systems mentioned herein may utilize any suitable number of subsystems.
  • a computer system includes a single computer apparatus, where the subsystems can be components of the computer apparatus.
  • a computer system can include multiple computer apparatuses, each being a subsystem, with internal components.
  • a computer system can include a plurality of the components or subsystems, e.g., connected together by external interface or by an internal interface.
  • computer systems, subsystems, or apparatuses can communicate over a network.
  • one computer can be considered a client and another computer a server, where each can be part of a same computer system.
  • a client and a server can each include multiple systems, subsystems, or components.
  • any of the embodiments of the present disclosure can be implemented in the form of control logic using hardware (e.g., an application specific integrated circuit or field programmable gate array) and/or using computer software with a generally programmable processor in a modular or integrated manner.
  • a processor includes a single-core processor, multi-core processor on a same integrated chip, or multiple processing units on a single circuit board or networked.
  • a person of ordinary skill in the art will know and appreciate other ways and/or methods to implement embodiments of the present disclosure using hardware and a combination of hardware and software.
  • Any of the software components or functions described in this application may be implemented as software code to be executed by a processor using any suitable computer language such as, for example, Java, C, C++, C#, Objective-C, Swift, or scripting language such as Perl or Python using, for example, conventional or object-oriented techniques.
  • the software code may be stored as a series of instructions or commands on a computer readable medium for storage and/or transmission, suitable media include random access memory (RAM), a read only memory (ROM), a magnetic medium such as a hard-drive or a floppy disk, or an optical medium such as a compact disk (CD) or DVD (digital versatile disk), flash memory, and the like.
  • RAM random access memory
  • ROM read only memory
  • magnetic medium such as a hard-drive or a floppy disk
  • an optical medium such as a compact disk (CD) or DVD (digital versatile disk), flash memory, and the like.
  • the computer readable medium may be any combination of such storage or transmission devices.
  • Such programs may also be encoded and transmitted using carrier signals adapted for transmission via wired, optical, and/or wireless networks conforming to a variety of protocols, including the Internet.
  • a computer readable medium according to an embodiment of the present disclosure may be created using a data signal encoded with such programs.
  • Computer readable media encoded with the program code may be packaged with a compatible device or provided separately from other devices (e.g., via Internet download). Any such computer readable medium may reside on or within a single computer product (e.g., a hard drive, a CD, or an entire computer system), and may be present on or within different computer products within a system or network.
  • a computer system may include a monitor, printer, or other suitable display for providing any of the results mentioned herein to a user.
  • Any of the methods described herein may be totally or partially performed with a computer system including one or more processors, which can be configured to perform the steps.
  • embodiments involve computer systems configured to perform the steps of any of the methods described herein, potentially with different components performing a respective steps or a respective group of steps.
  • steps of methods herein can be performed at a same time or in a different order. Additionally, portions of these steps may be used with portions of other steps from other methods.
  • 38 KILPATRICK TOWNSEND 780676821 all or portions of a step may be optional. Additionally, and of the steps of any of the methods can be performed with modules, circuits, or other means for performing these steps. 39 KILPATRICK TOWNSEND 780676821

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Computational Mathematics (AREA)
  • Algebra (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

Un modèle à deux étages selon l'invention permet d'augmenter l'équité de modèles d'apprentissage automatique malgré un accès limité à des catégories d'informations (par exemple, des attributs protégés) dans les données sous-jacentes sur lesquelles les modèles sont entraînés. Un premier étage est utilisé pour entraîner un modèle d'estimation d'attributs protégés par apprentissage actif. Un second étage est utilisé pour compléter des données ne contenant pas d'attributs protégés avec des estimations provenant du modèle d'estimation d'attributs protégés appris et permettre à un modèle d'atténuation de biais d'être entraîné avec un biais moindre.
PCT/US2023/086020 2023-06-09 2023-12-27 Équité d'apprentissage automatique avec attributs protégés limités Pending WO2024253705A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202363507381P 2023-06-09 2023-06-09
US63/507,381 2023-06-09

Publications (1)

Publication Number Publication Date
WO2024253705A1 true WO2024253705A1 (fr) 2024-12-12

Family

ID=93796369

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2023/086020 Pending WO2024253705A1 (fr) 2023-06-09 2023-12-27 Équité d'apprentissage automatique avec attributs protégés limités

Country Status (1)

Country Link
WO (1) WO2024253705A1 (fr)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200184350A1 (en) * 2018-12-10 2020-06-11 International Business Machines Corporation Post-hoc improvement of instance-level and group-level prediction metrics
CN113052383A (zh) * 2021-03-29 2021-06-29 上海酷量信息技术有限公司 一种基于机器学习的收入预测方法及装置
US20220101062A1 (en) * 2020-09-07 2022-03-31 Deutsche Telekom Ag. System and a Method for Bias Estimation in Artificial Intelligence (AI) Models Using Deep Neural Network

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200184350A1 (en) * 2018-12-10 2020-06-11 International Business Machines Corporation Post-hoc improvement of instance-level and group-level prediction metrics
US20220101062A1 (en) * 2020-09-07 2022-03-31 Deutsche Telekom Ag. System and a Method for Bias Estimation in Artificial Intelligence (AI) Models Using Deep Neural Network
CN113052383A (zh) * 2021-03-29 2021-06-29 上海酷量信息技术有限公司 一种基于机器学习的收入预测方法及装置

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
BELITZ K.; STACKELBERG P.E.: "Evaluation of six methods for correcting bias in estimates from ensemble tree machine learning regression models", ENVIRONMENTAL MODELLING & SOFTWARE, ELSEVIER, AMSTERDAM, NL, vol. 139, 24 February 2021 (2021-02-24), AMSTERDAM, NL , XP086542006, ISSN: 1364-8152, DOI: 10.1016/j.envsoft.2021.105006 *
ZHENPENG CHEN; JIE M. ZHANG; FEDERICA SARRO; MARK HARMAN: "A Comprehensive Empirical Study of Bias Mitigation Methods for Software Fairness", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 7 July 2022 (2022-07-07), 201 Olin Library Cornell University Ithaca, NY 14853, XP091265247 *

Similar Documents

Publication Publication Date Title
CN110929840B (zh) 使用滚动窗口的连续学习神经网络系统
US11551026B2 (en) Dynamic reconfiguration training computer architecture
US12288140B2 (en) Classifier assistance using domain-trained embedding
US10311368B2 (en) Analytic system for graphical interpretability of and improvement of machine learning models
US20240394574A1 (en) Determining bias-causing features of a machine learning model
WO2022072896A1 (fr) Réseaux graphiques probabilistes
Agarwal et al. Sentiment analysis in stock price prediction: A comparative study of algorithms
CN112257841B (zh) 图神经网络中的数据处理方法、装置、设备及存储介质
US20220083739A1 (en) Machine learning for joint recognition and assertion regression of elements in text
US20210357729A1 (en) System and method for explaining the behavior of neural networks
Chopra et al. Uncovering Semantic Inconsistencies and Deceptive Language in False News Using Deep Learning and NLP Techniques for Effective Management
WO2020190951A1 (fr) Réseau neuronal appris par augmentation homographique
US11593619B2 (en) Computer architecture for multiplier-less machine learning
Charanya et al. Customer Churn Prediction Analysis for Retention Using Ensemble Learning
JP2023544145A (ja) アクションへの傾向についての表現特徴空間における自己適応マルチモデル方法
US20230260506A1 (en) Adversarial language imitation with constrained exemplars
WO2024253705A1 (fr) Équité d'apprentissage automatique avec attributs protégés limités
Mácha et al. Deeptoppush: Simple and scalable method for accuracy at the top
Kaviya et al. Detecting Deceptive Identities: A Machine Learning Approach to Unveiling Fake Profiles on Social Media
Ahmad et al. A Data-Driven Approach for Online Phishing Activity Detection
Huang et al. Anchor attention for hybrid crowd forecasts aggregation
Boonyopakorn et al. Classifying Cybercrime and Threat on Thai Online News: A Comparison of Supervised Learning Algorithms
Mhsnhasan et al. Anomaly Detection in Social Networks using Improved Local Outlier Factor
US20250117513A1 (en) Data privacy using quick response code
US11922311B2 (en) Bias mitigating machine learning training system with multi-class target

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23940924

Country of ref document: EP

Kind code of ref document: A1