{"title": "Fairness Through Computationally-Bounded Awareness", "book": "Advances in Neural Information Processing Systems", "page_first": 4842, "page_last": 4852, "abstract": "We study the problem of fair classification within the versatile framework of Dwork et al. [ITCS '12], which assumes the existence of a metric that measures similarity between pairs of individuals.  Unlike earlier work, we do not assume that the entire metric is known to the learning algorithm; instead, the learner can query this *arbitrary* metric a bounded number of times.  We propose a new notion of fairness called *metric multifairness* and show how to achieve this notion in our setting.\nMetric multifairness is parameterized by a similarity metric d on pairs of individuals to classify and a rich collection C of (possibly overlapping) \"comparison sets\" over pairs of individuals.  At a high level, metric multifairness guarantees that *similar subpopulations are treated similarly*, as long as these subpopulations are identified within the class C.", "full_text": "Fairness Through Computationally-Bounded\n\nAwareness\n\nMichael P. Kim\u21e4\nStanford University\n\nmpk@cs.stanford.edu\n\nOmer Reingold\u21e4\nStanford University\n\nreingold@stanford.edu\n\nGuy N. Rothblum\u2020\n\nWeizmann Institute of Science\nrothblum@alum.mit.edu\n\nAbstract\n\nWe study the problem of fair classi\ufb01cation within the versatile framework of\nDwork et al. [6], which assumes the existence of a metric that measures similarity\nbetween pairs of individuals. Unlike earlier work, we do not assume that the\nentire metric is known to the learning algorithm; instead, the learner can query\nthis arbitrary metric a bounded number of times. We propose a new notion of\nfairness called metric multifairness and show how to achieve this notion in our\nsetting. Metric multifairness is parameterized by a similarity metric d on pairs of\nindividuals to classify and a rich collection C of (possibly overlapping) \u201ccomparison\nsets\" over pairs of individuals. At a high level, metric multifairness guarantees that\nsimilar subpopulations are treated similarly, as long as these subpopulations are\nidenti\ufb01ed within the class C.\n\n1\n\nIntroduction\n\nMore and more, machine learning systems are being used to make predictions about people. Algo-\nrithmic predictions are now being used to answer questions of signi\ufb01cant personal consequence; for\ninstance, Is this person likely to repay a loan? [24] or Is this person likely to recommit a crime?\n[1]. As these classi\ufb01cation systems have become more ubiquitous, concerns have also grown that\nclassi\ufb01ers obtained via machine learning might discriminate based on sensitive attributes like race,\ngender, or sexual orientation. Indeed, machine-learned classi\ufb01ers run the risk of perpetuating or\namplifying historical biases present in the training data. Examples of discrimination in classi\ufb01cation\nhave been well-illustrated [24, 1, 5, 19, 13, 3]; nevertheless, developing a systematic approach to\nfairness has been challenging. Often, it feels that the objectives of fair classi\ufb01cation are at odds with\nobtaining high-utility predictions.\nIn an in\ufb02uential work, Dwork et al. [6] proposed a framework to resolve the apparent con\ufb02ict\nbetween utility and fairness, which they call \u201cfairness through awareness.\" This framework takes the\nperspective that a fair classi\ufb01er should treat similar individuals similarly. The work formalizes this\nabstract goal by assuming access to a task-speci\ufb01c similarity metric d on pairs of individuals. The\nproposed notion of fairness requires that if the distance between two individuals is small, then the\npredictions of a fair classi\ufb01er cannot be very different. More formally, for some small constant \u2327  0,\nwe say a hypothesis f : X! [1, 1] satis\ufb01es (d, \u2327 )-metric fairness1 if the following (approximate)\nLipschitz condition holds for all pairs of individuals from the population X .\n|f (x)  f (x0)|\uf8ff d(x, x0) + \u2327\n(1)\nSubject to these intuitive similarity constraints, the classi\ufb01er may be chosen to maximize utility. Note\nthat, in general, the metric may be designed externally (say, by a regulatory agency) to address legal\n\n8x, x0 2X\u21e5X :\n\n\u21e4Supported by NSF Grant CCF-1763299.\n\u2020Supported by ISF grant No. 5219/17.\n1Note the de\ufb01nition given in [6] is slightly different; in particular, they propose a more general Lipschitz\n\ncondition, but \ufb01x \u2327 = 0.\n\n32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Montr\u00e9al, Canada.\n\n\fand ethical concerns, independent from the task of learning. In particular, in certain settings, the\nmetric designers may have access to a different set of features than the learner. For instance, perhaps\nthe metric designers have access to sensitive attributes, but for legal, social, or pragmatic reasons, the\nlearner does not. In addition to its conceptual simplicity, the modularity of fairness through awareness\nmakes it a very appealing framework. Currently, there are many (sometimes contradictory) notions of\nwhat it means for a classi\ufb01er to be fair [19, 5, 9, 13, 14], and there is much debate on which de\ufb01nitions\nshould be applied in a given context. Discrimination comes in many forms and classi\ufb01cation is used\nin a variety of settings, so naturally, it is hard to imagine any universally-applicable de\ufb01nition of\nfairness. Basing fairness on a similarity metric offers a \ufb02exible approach for formalizing a variety of\nguarantees and protections from discrimination.\nStill, a challenging aspect of this approach is the assumption that the similarity metric is known\nfor all pairs of individuals.2 Deciding on an appropriate metric is itself a delicate matter and could\nrequire human input from sociologists, legal scholars, and specialists with domain expertise. For\ninstance, in the loan repayment example, a simple, seemingly-objective metric might be a comparison\nof credit scores. A potential concern, however, is that these scores might themselves be biased\n(i.e. encode historical discriminations). In this case, a more nuanced metric requiring human input\nmay be necessary. Further, if the metric depends on features that are latent to the learner (e.g. some\nmissing sensitive feature) then the metric could appear arbitrarily complex to the learner. As such,\nin many realistic settings, the resulting metric will not be a simple function of the learner\u2019s feature\nvectors of individuals.\nIn most machine learning applications, where the universe of individuals is assumed to be very\nlarge, even writing down an appropriate metric could be completely infeasible. In these cases, rather\nthan require the metric value to be speci\ufb01ed for all pairs of individuals, we could instead ask a\npanel of experts to provide similarity scores for a small sample of pairs of individuals. While it is\ninformation-theoretically impossible to guarantee metric fairness from a sampling-based approach,\nwe still might hope to provide a strong, provable notion of fairness that maintains the theoretical\nappeal and practical modularity of the fairness through awareness framework.\nIn this work, we propose a new theoretical framework for fair classi\ufb01cation based on fairness through\nawareness \u2013 which we dub \u201cfairness through computationally-bounded awareness\u201d \u2013 that eliminates\nthe considerable issue of requiring the metric to be known exactly. Our approach maintains the\nsimplicity and \ufb02exibility of fairness through awareness, but provably only requires a small number of\nrandom samples from the underlying metric, even though we make no structural assumptions about the\nmetric. In particular, our approach works even if the metric provably cannot be learned. Speci\ufb01cally,\nour notion will require that a fair classi\ufb01er treat similar subpopulations of individuals similarly, in a\nsense we will make formal next. While our de\ufb01nition relaxes fairness through awareness, we argue\nthat it still protects against important forms of discrimination that the original work aimed to combat;\nfurther, we show that stronger notions necessarily require a larger sample complexity from the metric.\nAs in [6], we investigate how to learn a classi\ufb01er that achieves optimal utility under similarity-based\nfairness constraints, assuming a weaker model of limited access to the metric. We give positive and\nnegative results that show connections between achieving our fairness notion and learning.\n\n2 Setting and metric multifairness\n\nNotation. Let X\u2713 Rn denote the universe over individuals we wish to classify, where x 2X\nencodes the features of an individual. Let D denote the data distribution over individuals and labels\nsupported on X \u21e5 {1, 1}; denote by x, y \u21e0D a random draw from this distribution. Additionally,\nlet M denote the metric sample distribution over pairs of individuals. For a subset S \u2713X\u21e5X ,\nwe denote by (x, x0) \u21e0 S a random draw from the distribution M conditioned on (x, x0) 2 S. Let\nd : X\u21e5X! [0, 2] denote the underlying fairness metric that maps pairs to their associated distance.3\nOur learning objective will be to minimize the expectation over D of some convex loss function\nL : [1, 1] \u21e5 [1, 1] ! R+ over a convex hypothesis class F, subject to the fairness constraints.\nWe focus on agnostically learning the hypothesis class of linear functions with bounded weight; for\nsome constant B > 0, let F = [B, B]n. For w 2F , de\ufb01ne fw(x) = hw, xi, projecting fw(x)\n\n2Indeed, [6] identi\ufb01es this assumption as \u201cone of the most challenging aspects\u201d of the framework.\n3In fact, all of our results hold for a more general class of non-negative symmetric distance functions.\n\n2\n\n\fonto [1, 1] to get a valid prediction. We assume kxk1 \uf8ff 1 for all x 2X ; this is without loss of\ngenerality, by normalizing and increasing B appropriately.\nThe focus on linear functions is not too restrictive; in particular, by increasing the dimension to\nn0 = O(nk), we can learn any degree-k polynomial function of the original features. By increasing\nk, we can approximate increasingly complex functions.\n\n2.1 Metric multifairness\nWe de\ufb01ne our relaxation of metric fairness with respect to a rich class of statistical tests on the pairs\nof individuals. Let a comparison be any subset of the pairs of X\u21e5X . Our de\ufb01nition, which we call\nmetric multifairness, is parameterized by a collection of comparisons C\u2713 2X\u21e5X and requires that a\nhypothesis appear Lipschitz according to all of the statistical tests de\ufb01ned by the comparisons S 2C .\nDe\ufb01nition (Metric multifairness). Let C\u2713 2X\u21e5X be a collection of comparisons and let d : X\u21e5X !\n[0, 2] be a metric. For some constants \u2327  0, a hypothesis f is (C, d,\u2327 )-metric multifair if for all\nS 2C ,\n(2)\n\nE\n\n(x,x0)\u21e0S\u21e5|f (x)  f (x0)|\u21e4 \uf8ff E\n\n(x,x0)\u21e0S\u21e5d(x, x0)\u21e4 + \u2327.\n\nTo begin, note that metric multifairness is indeed a relaxation of metric fairness; if we take the\ncollection C = {{(x, x0)} : x, x0 2X\u21e5X}\nto be the collection of all pairwise comparisons, then\n(C, d,\u2327 )-metric multifairness is equivalent to (d, \u2327 )-metric fairness.\nIn order to achieve metric multifairness from a small sample from the metric, however, we need a\nlower bound on the density of each comparison in C; in particular, we can\u2019t hope to enforce metric\nfairness from a small sample. For some > 0, we say that a collection of comparisons C is -large if\nfor all S 2C , Pr(x,x0)\u21e0M[(x, x0) 2 S]  . A natural next choice for C would be a collection of\ncomparisons that represent the Cartesian products between traditionally-protected groups, de\ufb01ned by\nrace, gender, etc. In this case, as long as the minority populations are not too small, then a random\nsample from the metric will concentrate around the true expectation, and we could hope to enforce\nthis statistical relaxation of metric fairness. While this approach is information-theoretically feasible,\nits protections are very weak.\nTo highlight this weakness, suppose we want to predict the probability individuals will repay a loan,\nand our metric is an adjusted credit score. Even after adjusting scores, two populations P, Q \u2713X\n(say, de\ufb01ned by race) may have large average distance because overall P has better credit than Q; still,\nwithin P and Q, there may be signi\ufb01cant subpopulations P 0 \u2713 P and Q0 \u2713 Q that should be treated\nsimilarly (possibly representing the quali\ufb01ed members of each group). In this case, a coarse statistical\nrelaxation of metric fairness will not require that a classi\ufb01er treat P 0 and Q0 similarly; instead,\nthe classi\ufb01er could treat everyone in P better than everyone in Q \u2013 including treating unquali\ufb01ed\nmembers of P better than quali\ufb01ed members of Q. Indeed, the weaknesses of broad-strokes statistical\nde\ufb01nitions served as motivation for the original work of [6]. We would like to choose a class C\nthat strengthens the fairness guarantees of metric multifairness, but maintains its ef\ufb01cient sample\ncomplexity.\n\nComputationally-bounded awareness. While we can de\ufb01ne metric multifairness with respect to\nany collection C, typically, we will think of C as a rich class of overlapping subsets; equivalently,\nwe can think of the collection C as an expressive class of boolean functions, where for S 2C ,\ncS(x, x0) = 1 if and only if (x, x0) 2 S. In particular, C should be much more expressive than simply\nde\ufb01ning comparisons across traditionally-protected groups. The motivation for choosing such an\nexpressive class C is exempli\ufb01ed in the following proposition.\nProposition 1. Suppose there is some S 2C , such that E(x,x0)\u21e0S[d(x, x0)] \uf8ff \". Then if f is\n(C, d,\u2327 )-metric multifair, then f satis\ufb01es (d, (\" + \u2327 )/p)-metric fairness for at least a (1 p)-fraction\nof the pairs in S.\nThat is, if there is some subset S 2C that identi\ufb01es a set of pairs whose metric distance is small, then\nany metric multifair hypothesis must also satisfy the stronger individual metric fairness notion on many\npairs from S. This effect will compound if many different (possibly overlapping) comparisons are\nidenti\ufb01ed that have small average distance. We emphasize that these small-distance comparisons are\nnot known before sampling from the metric; indeed, this would imply the metric was (approximately)\nknown a priori. Still, if the class C is rich enough to correlate well with various comparisons that\n\n3\n\n\freveal signi\ufb01cant information about the metric, then any metric multifair hypothesis will satisfy\nindividual-level fairness on a signi\ufb01cant fraction of the population!\nWhile increasing the expressiveness of C increases the strength of the fairness guarantee, in order to\nlearn from a small sample, we cannot choose C to be arbitrarily complex. Thus, in choosing C we\nmust balance the strength of the fairness guarantee with the information bottleneck in accessing d\nthrough random samples. Our resolution to these competing needs is complexity-theoretic: while\ninformation-theoretically, we can\u2019t hope to ensure fair treatment across all subpopulations, we can\nhope ensure fair treatment across ef\ufb01ciently-identi\ufb01able subpopulations. For instance, if we take C\nto be a family de\ufb01ned according to some class of computations of bounded dimension \u2013 think, the\nset of conjunctions of a constant number of boolean features or short decision trees \u2013 then we can\nhope to accurately estimate and enforce the metric multifairness conditions. Taking such a bounded C\nensures that a hypothesis will be fair on all comparisons identi\ufb01able within this computational bound.\nThis is the sense in which metric multifairness provides fairness through computationally-bounded\nawareness.\n\n2.2 Learning model\n\nMetric access. Throughout, our goal is to learn a hypothesis from noisy samples from the metric\nthat satis\ufb01es multifairness. Speci\ufb01cally, we assume an algorithm can obtain a small number of\nindependent random metric samples (x, x0, (x, x0)) 2X\u21e5X\u21e5 [0, 2] where (x, x0) \u21e0M is drawn\nat random over the distribution of pairs of individuals, and (x, x0) is a random variable of bounded\nmagnitude with E[(x, x0)] = d(x, x0).\nWe emphasize that this is a very limited access model. As Theorem 2 shows we achieve (C, d,\u2327 )-\nmetric multifairness from a number of samples that depends logarithmically on the size of C indepen-\ndent of the complexity of the similarity metric.4 Recall that d : X\u21e5X! [0, 2] can be an arbitrary\nsymmetric function; thus, the learner does not necessarily have enough information to learn d. Still,\nfor exponentially-sized C, the learner can guarantee metric multifairness from a polynomial-sized\nsample, and the strength of the guarantee will scale up with the complexity of C (as per Propsition 1).\nIn order to ensure a strong notion of fairness, we assume that the subpopulations we wish to protect\nare well-represented in the pairs drawn from M. This assumption, while important, is not especially\nrestrictive, as we think of the metric samples as coming from a regulatory committee or ethically-\nmotivated party; in other words, in practical settings, it is reasonable to assume that one can choose\nthe metric sampling distribution based on the notion of fairness one wishes to enforce.\n\nLabel access. When we learn linear families, our goal will be to learn from a sample of labeled\nexamples. We assume the algorithm can ask for independent random samples x, y \u21e0D .\nMeasuring optimality. To evaluate the utility guarantees of our learned predictions, we take a\ncomparative approach. Suppose H\u2713 2X\u21e5X is a collection of comparisons. For \"  0, we say a\nhypothesis f is (H,\" )-optimal with respect to F, if\n[L(f (x), y)] \uf8ff E\nx,y\u21e0D\n\n[L(f\u21e4(x), y)] + \"\n\n(3)\n\nE\n\nx,y\u21e0D\n\nwhere f\u21e4 2F is an optimal (H, d, 0)-metric multifair hypothesis.\n3 Learning a metric multifair hypothesis\n\nAs in [6], we formulate the problem of learning a fair set of predictions as a convex program. Our\nobjective is to minimize the expected loss Ex,y\u21e0D[L(f (x), y)], subject to the multifairness constraints\nde\ufb01ned by C.5 Speci\ufb01cally, we show that a simple variant of stochastic gradient descent due to [20]\nlearns such linear families ef\ufb01ciently.\n\ndimension, metric entropy, etc.) through a uniform convergence argument.\n\n4Alternatively, for continuous classes of C, we can replace log(|C|) with some notion of dimension (VC-\n5For the sake of presentation, throughout the theorem statements, we will assume that L is O(1)-Lipschitz\n\non the domain of legal predictions/labels to guarantee bounded error; our results are proved more generally.\n\n4\n\n\f\u2327 2\n\nTheorem 2. Suppose , \u2327,  > 0 and C\u2713 2X\u21e5X is -large. With probability at least 1  ,\nstochastic switching subgradient descent learns a hypothesis w 2F that is (C, d,\u2327 )-metric multifair\nand (C, O(\u2327 ))-optimal with respect to F in O\u21e3 B2n2 log(n/)\n\u2318 iterations from m = \u02dcO\u21e3 log(|C|/)\n\u2318\nmetric samples. Each iteration uses at most 1 labeled example and can be implemented in\n\u02dcO (|C| \u00b7 n \u00b7 poly(1/, 1/\u2327, log(1/))) time.\nNote that the metric sample complexity depends logarithmically on |C|. Thus, information-\ntheoretically, we can hope to enforce metric mutlifairness with a class C that grows exponentially and\nstill be ef\ufb01cient. While the running time of each iteration depends on |C|, note that the number of\niterations is independent of |C|. In Section 4, we show conditions on |C| under which we can speed\nup the running time of each iteration to depend logarithmically on |C|.\nWe give a description of the switching subgradient method in Algorithm 1. At a high level, at each\niteration, the procedure checks to see if any constraint is signi\ufb01cantly violated. If it \ufb01nds a violation,\nit takes a (stochastic) step towards feasibility. Otherwise, it steps according a stochastic subgradient\nfor the objective.\nFor convenience of analysis, we de\ufb01ne the residual on the constraint de\ufb01ned by S as follows.\n\n\u2327 2\n\nRS(w) = E\n\n(x,x0)\u21e0S\u21e5|fw(x)  fw(x0)|\u21e4  E\n\n(x,x0)\u21e0S\u21e5d(x, x0)\u21e4\n\nNote that RS(w) is convex in the predctions fw(x) and thus, for linear families is convex in w.\nWe describe the algorithm assuming access to the following estimators, which we can implement\nef\ufb01ciently (in terms of time and samples). First, we assume we can estimate the residual \u02c6RS(w) on\n\neach S 2C with tolerance \u2327 such that for all w 2F ,RS(w)  \u02c6RS(w) \uf8ff \u2327. Next, we assume\na vector-valued random variable where E[r(w)w] 2 @(w). We assume access to stochastic\n\naccess to a stochastic subgradient oracle for the constraints and the objective. For a function (w),\nlet @(w) denote the set of subgradients of  at w. We abuse notation, and let r(w) refer to\nsubgradients for @RS(w) for all S 2C and @L(w). We include a full analysis of the algorithm and\nproof of Theorem 2 in the Appendix.\n\n(4)\n\n3.1 Post-processing for metric multifairness\nOne speci\ufb01c application of this result is as a way post-process learned predictions to ensure fairness.\nIn particular, suppose we are given the predictions from some pre-trained model for N individuals,\nbut are concerned that these predictions may not be fair. We can use Algorithm 1 to post-process these\nlabels to select near-optimal metric multifair predictions. Note these predictions will be optimal with\nrespect to the unconstrained family of predictions \u2013 not just predictions that come from a speci\ufb01c\nhypothesis class (like linear functions).\nSpeci\ufb01cally, in this setting we can represent an unconstrained set of predictions as a linear hypothesis\nin N dimensions: take B = 1, and let the feature vector for xi 2X be the ith standard basis vector.\nAlgorithm 1: Switching Subgradient Descent\nLet \u2327> 0, T 2 N, and C\u2713 2X\u21e5X .\nInitialize w0 2F = [B, B]n; W = ;\nFor k = 1, . . . , T :\n\n\u2022 If 9S 2C such that \u02c6RS(wk) > 4\u2327/ 5:\n\n\u2013 Sk any S 2C such that \u02c6RS(wk) > 4\u2327/ 5\n\u2013 wk+1 wk  \u2327\n\nM 2rRSk (wk)\n\n// some constraint violated\n\n/* step according to constraint\nproject onto F if necessary */\n// no violations found\n// update set of feasible iterates\n/* step according to objective\nproject onto F if necessary */\n// output average of feasible iterates\n\n\u2022 Else:\n\n\u2013 W W [{ wk}\n\u2013 wk+1 wk  \u2327\n|W| \u00b7Pw2W w\n\nGM rL(wk)\n\nOutput \u00afw = 1\n\n5\n\n\fThen, we can think of the input labels to Algorithm 1 to be the output of any predictor that was trained\nseparately.6 For instance, if we have learned a highly-accurate model, but are unsure of its fairness,\nwe can instantiate our framework with, say, the squared loss between the original predictions and the\nreturned predictions; then, we can view the program as a procedure to project the highly-accurate\npredictions onto the set of metric multifair predictions. Importantly, our procedure only needs a small\nset of samples from the metric and not the original data used to train the model.\nPost-processing prediction models for fairness has been studied in a few contexts [25, 14, 18]. This\npost-processing setting should be contrasted to these settings. In our setting, the predictions are\nnot required to generalize out of sample (in terms of loss or fairness). On the one hand, this means\nthe metric multifairness guarantee does not generalize outside the N individuals; on the other hand,\nbecause the predictions need not come from a bounded hypothesis class, their utility can only improve\ncompared to learning a metric multifair hypothesis directly.\nIn addition to preserving the utility of previously-trained classi\ufb01ers, separating the tasks of training\nfor utility and enforcing fairness may be desirable when intentional malicious discrimination may\nbe anticipated. For instance, when addressing the forms of racial pro\ufb01ling that can occur through\ntargeted advertising (as described in [6]), we may not expect self-interested advertisers to adhere\nto classi\ufb01cation under strict fairness constraints, but it stands to reason that prominent advertising\nplatforms might want to prevent such blatant abuses of their platform. In this setting, the platform\ncould impose metric multifairness after the advertisers specify their ideal policy.\n\n4 Reducing search to agnostic learning\n\nAs presented above, the switching subgradient descent method converges to a nearly-optimal point\nin a bounded number of subgradient steps, independent of |C|. The catch is that at the beginning of\neach iteration, we need to search over C to determine if there is a signi\ufb01cantly violated multifairness\nconstraint. As we generally want to take C to be a rich class of comparisons, in many cases |C| will\nbe prohibitive. As such, we would hope to \ufb01nd violated constraints in sublinear time, preferably\neven poly-logarithmic in |C|. Indeed, we show that if a concept class C admits an ef\ufb01cient agnostic\nlearner, then we can solve the violated constraint search problem over the corresponding collection of\ncomparisons ef\ufb01ciently.\nAgnostic learning can be phrased as a problem of detecting correlations. Suppose g, h : U! [1, 1],\nand let D be some distribution supported on U. We denote the correlation between g and h on D as\nhg, hi = Ei\u21e0D[g(i) \u00b7 h(i)]. We let C\u2713 [1, 1]U denote the concept class and H\u2713 [1, 1]U denote\nthe hypothesis class. The task of agnostic learning can be stated as follows: given sample access\nover some distribution (i, g(i)) \u21e0D\u21e5 [1, 1] to some function g 2 [1, 1]N, \ufb01nd some hypothesis\nh 2H that is comparably correlated with g as the best c 2C . That is, given access to g, an agnostic\nlearner with accuracy \" for concept class C returns some h from the hypothesis class H such that\n\nhg, hi + \"  max\n\nc2C hg, ci.\n\n(5)\n\nAn agnostic learner is typically considered ef\ufb01cient if it runs in polynomial time in log(|C|) (or an\nappropriate notion of dimension of C), 1/\", and log(1/d). Additionally, distribution-speci\ufb01c learners\nand learners with query access to the function have been studied [12, 7]. In particular, membership\nqueries tend to make agnostic learning easier. Our reduction does not use any metric samples other\nthan those that the agnostic learner requests. Thus, if we are able to query a panel of experts according\nto the learner, rather than randomly, then an agnostic learner that uses queries could be used to speed\nup our learning procedure.\nTheorem 3. Suppose there is an algorithm A for agnostic learning the concept class C with hy-\npothesis class H that achieves accuracy \" with probability 1   in time TA(\", ) from mA(\", )\nlabeled samples. Suppose that C is -large. Then, there is an algorithm that, given access to\nT = \u02dcO\u21e3 B2n2\n\u2327 2 \u2318 labeled examples, outputs a set of predictions that are (C, d,\u2327 )-metric multifair\nand (H, O(\u2327 ))-optimal with respect to F = [B, B]n that runs in time \u02dcO\u21e3 TA(\u2327,/T )\u00b7B2n2\n\u2318 and\nrequires m = \u02dcO\u21e3 log(|C|)\n\n\u2318 metric samples.\n\n6Nothing in our analysis required labels y 2{ 1, 1}; we can instead take the labels y 2 [1, 1].\n\n\u2327 2 + nA(\u2327,/T )\n\n\u2327 2\n\n2\u2327 2\n\n6\n\n\fWhen we solve the convex program with switching subgradient descent, at the beginning of each\niteration, we check if there is any S 2C such that the residual quantity RS(w) is greater than\nsome threshold. If we \ufb01nd such an S, we step according to the subgradient of RS(w). In fact,\nthe proof of the convergence of switching subgradient descent reveals that as long as when there\nis some S 2C where RS(w) is in violation, we can \ufb01nd some RS0(w) >\u21e2 for some constant \u21e2,\nwhere S0 2H\u2713 [1, 1]X\u21e5X , then we can argue that the learned hypothesis will be (C, d,\u2327 )-metric\nmultifair and achieve utility commensurate with the best (H, d, 0)-metric multifair hypothesis.\nWe show a general reduction from the problem of searching for a violated comparison S 2C to\nthe problem of agnostic learning over the corresponding family of boolean functions. In particular,\nrecall that for a collection of comparisons C\u2713 2X\u21e5X , we can also think of C as a family of boolean\nconcepts, where for each S 2C , there is an associated boolean function cS : X \u21e5 X ! {1, 1}\nwhere cS(xi, xj) = 1 if and only if (xi, xj) 2 S. We frame this search problem as an agnostic\nsuch that any\nlearning problem, where we design a set of \u201clabels\u201d for each pair (x, x0) 2X\u21e5X\nfunction that is highly correlated with these labels encodes a way to update the parameters towards\nfeasibility.\n\nProof. Recall the search problem: given a current hypothesis fw, is there some S 2C such that\nRS(w) = E(x,x0)\u21e0S[|fw(xi)  fw(xj)| d(xi, xj)] >\u2327 ? Consider the labeling each pair (xi, xj)\nwith v(xi, xj) = |fw(xi)  fw(xj)| d(xi, xj). Let \u21e2 = RX\u21e5X (w); note that we can treat \u21e2\nas a constant for all S 2C . Further, suppose S 2C is such that RS(w) >\u2327 , or equivalently,\nE(xi,xj )\u21e0S[v(xi, xj)] >\u2327 . Then, by the assumption that C is -large, the correlation between the\ncorresponding boolean function cS and labels v can be lower bounded as hcS, vi > 2\u2327  \u21e2. Suppose\naccuracy. Then, we know that\nwe have an agnostic learner that returns a hypothesis h with \"<\u2327\nhh, vi  \u2327  \u21e2 by the lower bound on the optimal cS 2C . Then, consider the function Rh(w)\nde\ufb01ned as follows.\n\n(x,x0)\u21e0X\u21e5X\uf8ff\u2713 h(x, x0) + 1\n\nE\n\n2\n\n\u25c6 \u00b7 v(x, x0)\n\nRSh(w) =\n\n= hh, vi + \u21e2\n\n2\n\n(6)\n\n(7)\n\nThus, given that there exists some S 2C where RS(w) >\u2327 , we can \ufb01nd some real-valued comparison\nSh(x, x0) = h(x,x0)+1\n\n, such that RSh(w) =\u2326( hh, vi + \u21e2)  \u2326(\u2327 ).\n\n2\n\nDiscovering a violation of at least \u2326(\u2327 ) guarantees \u2326(2\u2327 2) progress in the duality gap at each step,\nso the theorem follows from the analysis of Algorithm 1.\n\n5 Hardness of learning metric multifair hypotheses\n\nIn this section, we show that our algorithmic results cannot be improved signi\ufb01cantly. In particular,\nwe focus on the post-processing setting of Section 3.1. We show that the metric sample complexity is\ntight up to a \u2326(log log(|C|)) factor unconditionally. We also show that some learnability assumption\non C is necessary in order to achieve a high-utility (C, d,\u2327 )-metric multifair predictions ef\ufb01ciently. In\nparticular, we give a reduction from inverting a boolean concept c 2C to learning a hypothesis f\nthat is metric multifair on a collection H derived from C, where the metric samples from d encode\ninformation about the concept c. Recall, that for any H and d, we can always output a trivial\n(H, d, 0)-metric multifair hypothesis by outputting a constant hypothesis. This leads to a subtlety in\nour reductions, where we need to leverage the learner\u2019s ability to simultaneously satisfy the metric\nmultifairness constraints and achieve high utility.\nBoth lower bounds follow the same general construction. Suppose we have some boolean concept\nclass C \u2713 {1, 1}X0 for some universe X0. We will construct a new universe X = X0 [X 1 and\nde\ufb01ne a collection of \u201cbipartite\u201d comparisons over subsets of X0 \u21e5X 1. Then, given samples from\n(x0, c(x0)), we de\ufb01ne corresponding metric values where d(x0, x1) is some function of c(x0) for all\nx1 2X 1. Finally, we need to additionally encode the objective of inverting c into labels for x0 2X 0,\nsuch that to obtain good loss, the post-processor must invert c on X0. We give a full description of\nthe reduction in the Appendix.\n\n7\n\n\fLower bounding the sample complexity. While we argued earlier that some relaxation of metric\nfairness is necessary if we want to learn from a small set of metric samples, it is not clear that\nmultifairness with respect to C is the strongest relaxation we can obtain. In particular, we might\nhope to guarantee fairness on all large comparisons, rather than just a \ufb01nite class C. The following\ntheorem shows that such a hope is misplaced: in order for an algorithm to guarantee that the Lipschitz\ncondition holds in expectation over a \ufb01nite collection of large comparisons C, then either the algorithm\ntakes \u2326(log |C|) random metric samples, or the algorithm outputs a set of nearly useless predictions.\nFor concreteness, we state the theorem in the post-processing setting of Section 3.1; the construction\ncan be made to work in the learning setting as well.\nTheorem 4. Let , \u2327 > 0 be constants and suppose A is an algorithm that has random sample\naccess to d and outputs a (C, d,\u2327 )-metric multifair set of predictions for -large C. Then, A takes\n\u2326(log |C|) random samples from d or outputs a set of predictions with loss that approaches the loss\nachievable with no metric queries.\nThe construction uses a reduction from the problem of learning a linear function; we then appeal to a\nlower bound from linear algebra on the number of random queries needed to span a basis.\n\nHardness from pseudorandom functions. Our reduction implies that a post-processing algorithm\nfor (C, d,\u2327 )-metric multifairness with respect to an arbitrary metric d gives us a way of distinguishing\nfunctions in C from random.\nProposition 5 (Informal). Assuming one-way functions exist, there is no ef\ufb01cient algorithm for\ncomputing (C,\u2327 )-optimal (C, d,\u2327 )-metric multifair predictions for general C, d, and constant \u2327.\nEssentially, without assumptions that C is a learnable class of boolean functions, some nontrivial\nrunning time dependence on |C| is necessary. The connection between learning and pseudorandom\nfunctions [23, 11] is well-established; under stronger cryptographic assumptions as in [2], the\nreduction implies that a running time of \u2326(|C|\u21b5) is necessary for some constant \u21b5> 0.\n6 Related works and discussion\n\nMany exciting recent works have investigated fairness in machine learning. In particular, there is\nmuch debate on the very de\ufb01nitions of what it means for a classi\ufb01er to be fair [19, 4, 21, 13, 5, 14].\nBeyond the work of Dwork et al. [6], our work bears most similarity to two recent works of H\u00e9bert-\nJohnson et al. and Kearns et al. [14, 16]. As in this work, both of these papers investigate notions of\nfairness that aim to strengthen the guarantees of statistical notions, while maintaining their practicality.\nThese works also both draw connections between achieving notions of fairness and ef\ufb01cient agnostic\nlearning. In general, agnostic learning is considered a notoriously hard computational problem\n[15, 17, 8]; that said, in the context of fairness in machine learning, [16] show that using heuristic\nmethods to agnostically learn linear hypotheses seems to work well in practice.\nMetric multifairness does not directly generalize either [14] or [16], but we argue that it provides a\nmore \ufb02exible alternative to these approaches for subpopulation fairness. In particular, these works\naim to achieve speci\ufb01c notions of fairness \u2013 either calibration or equalized error rates \u2013 across a rich\nclass of subpopulations. As has been well-documented [19, 4, 21], calibration and equalized error\nrates, in general, cannot be simultaneously satis\ufb01ed. Often, researchers frame this incompatibility\nas a choice: either you satisfy calibration or you satisfy equalized error rates; nevertheless, there\nare many applications where some interpolation between accuracy (\u00e0 la calibration) and corrective\ntreatment (\u00e0 la equalized error rates) seems appropriate.\nMetric-based fairness offers a way to balance these con\ufb02icting fairness desiderata. In particular, one\ncould design a similarity metric that preserves accuracy in predictions and separately a metric that\nperforms corrective treatment, and then enforce metric multifairness on an appropriate combination\nof the metrics. For instance, returning to the loan repayment example, an ideal metric might be a\ncombination of credit scores (which tend to be calibrated) and a metric that aims to increase the\nloans given to historically underrepresented populations (by, say, requiring the top percentiles of\neach subpopulation be treated similarly). Different combinations of the two metrics would place\ndifferent weights on the degree of calibration and corrective discrimination in the resulting predictor.\nOf course, one could equally apply this metric in the framework of [6], but the big advantage with\nmetric multifairness is that we only need a small sample from the metric to provide a relaxed, but still\nstrong guarantee of fairness.\n\n8\n\n\fWe are optimistic that metric multifairness will provide an avenue towards implementing metric-based\nfairness notions. At present, the results are theoretical, but we hope this work can open the door to\nempirical studies across diverse domains, especially since one of the strengths of the framework is its\ngenerality. We view testing the empirical performance of metric multifairness with various choices of\nmetric d and collection C as an exciting direction for future research.\nFinally, two recent theoretical works also investigate extensions to the fairness through awareness\nframework of [6]. Gillen et al. [10] study metric-based individually fair online decision-making in\nthe presence of an unknown fairness metric. In their setting, every day, a decision maker must choose\nbetween candidates available on that day; the goal is to have the decision maker\u2019s choices appear\nmetric fair on each day (but not across days). Their work makes a strong learnability assumption\nabout the underlying metric; in particular, they assume that the unknown metric is a Mahalanobis\nmetric, whereas our focus is on fair classi\ufb01cation when the metric is unknown and unrestricted.\nRothblum and Yona [22] study fair machine learning under a different relaxation of metric fairness,\nwhich they call approximate metric fairness. They assume that the metric is fully speci\ufb01ed and known\nto the learning algorithm, whereas our focus is on addressing the challenge of an unknown metric.\nTheir notion of approximate metric fairness aims to protect all (large enough) groups, and thus, is\nmore strict than metric multifairness.\n\nAcknowledgements. The authors thank Cynthia Dwork, Roy Frostig, Fereshte Khani, Vatsal\nSharan, Paris Siminelakis, and Gregory Valiant for helpful conversations and feedback on earlier\ndrafts of this work. We thank the anonymous reviewers for their careful reading and suggestions on\nhow to improve the clarity of the presentation.\n\n9\n\n\fReferences\n[1] Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. Machine bias: There\u2019s software\nused across the country to predict future criminals. and it\u2019s biased against blacks. ProPublica,\n2016.\n\n[2] Andrej Bogdanov and Alon Rosen. Pseudorandom functions: Three decades later. In Tutorials\n\non the Foundations of Cryptography, pages 79\u2013158. Springer, 2017.\n\n[3] Joy Buolamwini and Timnit Gebru. Gender shades: Intersectional accuracy disparities in\ncommercial gender classi\ufb01cation. In Conference on Fairness, Accountability and Transparency,\npages 77\u201391, 2018.\n\n[4] Alexandra Chouldechova. Fair prediction with disparate impact: A study of bias in recidivism\n\nprediction instruments. Big Data, 2017.\n\n[5] Sam Corbett-Davies, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. Algorithmic\n\ndecision making and the cost of fairness. KDD, 2017.\n\n[6] Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard S. Zemel. Fairness\nthrough awareness. In Innovations in Theoretical Computer Science (ITCS), pages 214\u2013226,\n2012.\n\n[7] Vitaly Feldman. Distribution-speci\ufb01c agnostic boosting. In Proceedings of the First Symposium\n\non Innovations in Computer Science\u201910, 2010.\n\n[8] Vitaly Feldman, Venkatesan Guruswami, Prasad Raghavendra, and Yi Wu. Agnostic learning\n\nof monomials by halfspaces is hard. SIAM Journal on Computing, 41(6):1558\u20131590, 2012.\n\n[9] Avi Feller, Emma Pierson, Sam Corbett-Davies, and Sharad Goel. A computer program used\nfor bail and sentencing decisions was labeled biased against blacks. it\u2019s actually not that clear.\nThe Washington Post, 2016.\n\n[10] Stephen Gillen, Christopher Jung, Michael J. Kearns, and Aaron Roth. Online learning with an\n\nunknown fairness metric. arXiv preprint arXiv:1802.06936, 2018.\n\n[11] Oded Goldreich, Sha\ufb01 Goldwasser, and Silvio Micali. How to construct random functions. In\nFoundations of Computer Science, 1984. 25th Annual Symposium on, pages 464\u2013479. IEEE,\n1984.\n\n[12] Parikshit Gopalan, Adam Tauman Kalai, and Adam R Klivans. Agnostically learning decision\ntrees. In Proceedings of the fortieth annual ACM symposium on Theory of computing, pages\n527\u2013536. ACM, 2008.\n\n[13] Moritz Hardt, Eric Price, and Nathan Srebro. Equality of opportunity in supervised learning. In\n\nAdvances in Neural Information Processing Systems, pages 3315\u20133323, 2016.\n\n[14] \u00darsula H\u00e9bert-Johnson, Michael P. Kim, Omer Reingold, and Guy N. Rothblum. Calibration\n\nfor the (computationally-identi\ufb01able) masses. ICML, 2018.\n\n[15] Michael Kearns. Ef\ufb01cient noise-tolerant learning from statistical queries. Journal of the ACM\n\n(JACM), 45(6):983\u20131006, 1998.\n\n[16] Michael Kearns, Seth Neel, Aaron Roth, and Zhiwei Steven Wu. Preventing fairness gerryman-\n\ndering: Auditing and learning for subgroup fairness. ICML, 2018.\n\n[17] Michael J. Kearns, Robert E. Schapire, and Linda M. Sellie. Toward ef\ufb01cient agnostic learning.\n\nMachine Learning, 17(2-3):115\u2013141, 1994.\n\n[18] Michael P. Kim, Amirata Ghorbani, and James Zou. Multiaccuracy: Black-box post-processing\n\nfor fairness in classi\ufb01cation. arXiv preprint arXiv:1805.12317, 2018.\n\n[19] Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. Inherent trade-offs in the fair\n\ndetermination of risk scores. ITCS, 2017.\n\n10\n\n\f[20] Yurii Nesterov. Primal-dual subgradient methods for convex problems. Mathematical program-\n\nming, 120(1):221\u2013259, 2009.\n\n[21] Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, and Kilian Q. Weinberger. On\n\nfairness and calibration. NIPS, 2017.\n\n[22] Guy N. Rothblum and Gal Yona. Probably approximately metric-fair learning. ICML, 2018.\n[23] Leslie G. Valiant. A theory of the learnable. Communications of the ACM, 27(11):1134\u20131142,\n\n1984.\n\n[24] Kaveh Waddell. How algorithms can bring down minorities\u2019 credit scores. The Atlantic, 2016.\n[25] Blake Woodworth, Suriya Gunasekar, Mesrob I Ohannessian, and Nathan Srebro. Learning\n\nnon-discriminatory predictors. COLT, 2017.\n\n11\n\n\f", "award": [], "sourceid": 2341, "authors": [{"given_name": "Michael", "family_name": "Kim", "institution": "Stanford University"}, {"given_name": "Omer", "family_name": "Reingold", "institution": "Stanford University"}, {"given_name": "Guy", "family_name": "Rothblum", "institution": "Weizmann Institute of Science"}]}