{"title": "Differentially Private Testing of Identity and Closeness of Discrete Distributions", "book": "Advances in Neural Information Processing Systems", "page_first": 6878, "page_last": 6891, "abstract": "We study the fundamental problems of identity testing (goodness of fit), and closeness testing (two sample test) of distributions over $k$ elements, under differential privacy. While the problems have a long history in statistics,  finite sample bounds for these problems have only been established recently. \n\nIn this work, we derive upper and lower bounds on the sample complexity of both the problems under $(\\varepsilon, \\delta)$-differential privacy. We provide optimal sample complexity algorithms for identity testing problem for all parameter ranges, and the first results for closeness testing. Our closeness testing bounds are optimal in the sparse regime where the number of samples is at most $k$. \n\nOur upper bounds are obtained by privatizing non-private estimators for these problems. The non-private estimators are chosen to have small sensitivity. We propose a general framework to establish lower bounds on the sample complexity of statistical tasks under differential privacy. We show a bound on differentially private algorithms in terms of a coupling between the two hypothesis classes we aim to test. By constructing carefully chosen priors over the hypothesis classes, and using Le Cam's two point theorem we provide a general mechanism for proving lower bounds.  We believe that the framework can be used to obtain strong lower bounds for other statistical tasks under privacy.", "full_text": "Dierentially Private Testing of Identity and\n\nCloseness of Discrete Distributions\n\nJayadev Acharya \u00fa\nCornell University\n\nacharya@cornell.edu\n\nZiteng Sun \u00fa\n\nCornell University\nzs335@cornell.edu\n\nHuanyu Zhang \u00fa\nCornell University\nhz388@cornell.edu\n\nAbstract\n\nWe study the fundamental problems of identity testing (goodness of \ufb01t), and\ncloseness testing (two sample test) of distributions over k elements, under\ndierential privacy. While the problems have a long history in statistics,\n\ufb01nite sample bounds for these problems have only been established recently.\nIn this work, we derive upper and lower bounds on the sample complexity\nof both the problems under (\u00c1, \u201d)-dierential privacy. We provide sample\noptimal algorithms for identity testing problem for all parameter ranges,\nand the \ufb01rst results for closeness testing. Our closeness testing bounds are\noptimal in the sparse regime where the number of samples is at most k.\nOur upper bounds are obtained by privatizing non-private estimators for\nthese problems. The non-private estimators are chosen to have small sensi-\ntivity. We propose a general framework to establish lower bounds on the\nsample complexity of statistical tasks under dierential privacy. We show a\nbound on dierentially private algorithms in terms of a coupling between\nthe two hypothesis classes we aim to test. By carefully constructing chosen\npriors over the hypothesis classes, and using Le Cam\u2019s two point theorem we\nprovide a general mechanism for proving lower bounds. We believe that the\nframework can be used to obtain strong lower bounds for other statistical\ntasks under privacy.\n\n1 Introduction\nTesting whether observed data conforms to an underlying model is a fundamental scienti\ufb01c\nproblem. In a statistical framework, given samples from an unknown probabilistic model,\nthe goal is to determine whether the underlying model has a property of interest.\nThis question has received great attention in statistics as hypothesis testing [1, 2], where it\nwas mostly studied in the asymptotic regime when the number of samples m \u00e6 \u0152. In the\npast two decades there has been a lot of work from the computer science, information theory,\nand statistics community on various distribution testing problems in the non-asymptotic\n(small-sample) regime, where the domain size k could be potentially larger than m (See [3,\n4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15], references therein, and [16] for a recent survey). Here\nthe goal is to characterize the minimum number of samples necessary (sample complexity)\nas a function of the domain size k, and the other parameters.\nAt the same time, preserving the privacy of individuals who contribute to the data samples\nhas emerged as one of the key challenges in designing statistical mechanisms over the last few\nyears. For example, the privacy of individuals participating in surveys on sensitive subjects\n\u00faThe authors are listed in alphabetical order. This research was supported by NSF-CCF-CRII\n\n1657471, and a grant from Cornell University.\n\n32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Montr\u00e9al, Canada.\n\n\fis of utmost importance. Without a properly designed mechanism, statistical processing\nmight divulge the sensitive information about the data. There have been many publicized\ninstances of individual data being de-anonymized, including the deanonymization of Net\ufb02ix\ndatabase [17], and individual information from census-related data [18]. Protecting privacy\nfor the purposes of data release, or even computation on data has been studied extensively\nacross several \ufb01elds, including statistics, machine learning, database theory, algorithm design,\nand cryptography (See e.g., [19, 20, 21, 22, 23, 24, 25]). While the motivation is clear, even\na formal notion of privacy is not straight forward. We use dierential privacy [26], a notion\nwhich rose from database and cryptography literature, and has emerged as one of the most\npopular privacy measures (See [26, 27, 22, 28, 29, 30, 31, 32], references therein, and the\nrecent book [33]). Roughly speaking, it requires that the output of the algorithm should be\nstatistically close on two neighboring datasets. For a formal de\ufb01nition of dierential privacy,\nsee Section 2.\nA natural question when designing a dierentially private algorithm is to understand how\nthe data requirement grows to ensure privacy, along with the same accuracy. In this paper,\nwe study the sample size requirements for dierentially private discrete distribution testing.\n\n1.1 Results and Techniques\nWe consider two fundamental statistical tasks for testing distributions over [k]: (i) identity\ntesting, where given sample access to an unknown distribution p, and a known distribution\nq, the goal is to decide whether p = q, or dT V (p, q) \u00d8 \u2013, and (ii) closeness testing, where\ngiven sample access to unknown distributions p, and q, the goal is to decide whether p = q,\nor dT V (p, q) \u00d8 \u2013. (See Section 2 for precise statements of these problems). Given dierential\nprivacy constraints (\u00c1, \u201d), we provide (\u00c1, \u201d)-dierentially private algorithms for both these\ntasks. For identity testing, our bounds are optimal up to constant factors for all ranges of\nk, \u2013, \u00c1, \u201d , and for closeness testing the results are tight in the small sample regime where\nm = O(k). Our upper bounds are based on various methods to privatize the previously\nknown tests. A critical component is to design and analyze test statistic that have low\nsensitivity (see De\ufb01nition 4), in order to preserve privacy.\nWe \ufb01rst state that any (\u00c1 + \u201d, 0)-DP algorithm is also an (\u00c1, \u201d) algorithm. [34] showed that\nfor testing problems, any (\u00c1, \u201d) algorithm will also imply a (\u00c1 + c\u201d, 0)-DP algorithm. Please\nrefer to Lemma 2 and Lemma 3 for more detail. Therefore, for all the problems, we simply\nconsider (\u00c1, 0)-DP algorithms (\u00c1-DP), and we can replace \u00c1 with (\u00c1 + \u201d) in both the upper\nand lower bounds without loss of generality.\nOne of the main contributions of our work is to propose a general framework for establishing\nlower bounds for the sample complexity of statistical problems such as property estimation\nand hypothesis testing under privacy constraints. We describe this, and the other results\nbelow. A summary of the results is presented in Table 1, which we now describe in detail.\n1. DP Lower Bounds via Coupling. We establish a general method to prove lower\nbounds for distribution testing problems. Suppose X m1 , and Y m1 are generated by two\nstatistical sources. Further suppose there is a coupling between the two sources such\nthat the expected hamming distance between the coupled samples is at most D, then if\n\u00c1 + \u201d = o(1/D), there is no (\u00c1, \u201d)-dierentially private algorithm to distinguish between\nthe two sources. This result is stated precisely in Theorem 1. By carefully using designed\ncoupling schemes, we provide lower bounds for identity testing, and closeness testing.\n2. Reduction from identity to uniformity. We reduce the problem of \u00c1-DP identity\ntesting of distributions over [k] to \u00c1-DP uniformity testing over distributions over [6k].\nSuch a reduction, without privacy constraints was shown in [35], and we use their result\nto obtain a reduction that also preserves privacy, with at most a constant factor blow-up\nin the sample complexity. This result is given in Theorem 3.\n\n3. Identity Testing. It was recently shown that O(\u00d4k\n\n\u20132 ) [7, 36, 11, 37] samples are necessary\nand sucient for identity testing without privacy constraints. The statistic used in these\npapers are variants of chi-squared tests, which could have a high global sensitivity.\nGiven the reduction from identity to uniformity, it suces to consider uniformity testing.\nWe consider the test statistic studied by [38] which is simply the distance of the empirical\ndistribution to the uniform distribution. This statistic also has a low sensitivity, and\n\n2\n\n\ffuthermore has the optimal sample complexity in all parameter ranges, without privacy\nconstraints. In Theorem 2, we state the optimal sample complexity of identity testing.\nThe upper bounds are derived by privatizing the statistic in [38]. For lower bound, we use\nour technique in Theorem 1. We design a coupling between the uniform distribution u[k],\nand a mixture of distributions, which are all at distance \u2013 from u[k] in total variation\ndistance. In particular, we consider the mixture distribution used in [7]. Much of the\ntechnical details go into proving the existence of couplings with small expected Hamming\ndistance. [34] studied identity testing under pure dierential privacy, and obtained an\n\nbounds signi\ufb01cantly.\n\u00d4k\n\n4. Closeness Testing. Closeness testing problem was proposed by [3], and optimal bound\n\nalgorithm with complexity O3\u00d4k\nof 1max{ k2/3\nof 1 k2/3\n\u20134/3 + \u00d4k\n\n\u20135/3\u00c12/3 4. Our results improve their\n\u20132 }2 was shown in [10]. They proposed a chi-square based statistic,\n\u20132\u00c12. These\n\u2013\u00d4\u00c12, and in the dense regime, we obtain a bound of O1\u00d4k\n\nwhich we show has a small sensitivity. We privatize their algorithm to obtain the\nsample complexity bounds. In the sparse regime we prove a sample complexity bound\n\nresults are stated in Theorem 4. Since closeness testing is a harder problem than identity\ntesting, all the lower bounds from identity testing port over to closeness testing. The\ncloseness testing lower bounds are given in Theorem 4.\n\n\u20132 + \u00d4k log k\n\n\u20133/2\u00c1 + (k log k)1/3\n\n\u20132 + 1\n\n\u20134/3 ,\n\nProblem\n\nIdentity Testing\n\nCloseness Testing\n\nSample Complexity Bounds\n\n\u201322 [7]\nNon-private : 1\u00d4k\n\u20133/2\u00c1 4 [34]\n\u00c1-DP algorithms: O3\u00d4k\n\u20132 + \u00d4k log k\n\u20132 + max\u00d3 k1/2\nS(IT, k,\u2013,\u00c1 ) =1\u00d4k\nk1/3\n\u20134/3\u00c12/3 , 1\n\u2013\u00c11/2 ,\n\u20132 2 [10]\nNon-private: 1 k2/3\n\u20134/3 + k1/2\nIF \u20132 =1 1\u00d4k2 and \u20132\u00c1 =! 1\nk\"\n\u2013\u00d4\u00c12\nS(CT, k,\u2013,\u00c1 ) =1 k2/3\n\u20134/3 + \u00d4k\n\u2013\u00c12 \u00c6 S(CT, k,\u2013,\u00c1 ) \u00c6 O1\u00d4k\n1\u00d4k\n\u20132 + \u00d4k\n\n\u00c1-DP algorithms:\n\n\u2013\u00d4\u00c1 + 1\n\n\u20132 + 1\n\nELSE\n\n\u2013\u00c1\u00d42[Theorem 2]\n\n\u20132\u00c12 [Theorem 4]\n\nTable 1: Summary of the sample complexity bounds for \u00c1-DP identity, and closeness testing.\nFor (\u00c1, \u201d)-DP algorithms, we can simply replace \u00c1 in the sample complexity by (\u00c1 + \u201d).\n\n1.2 Related Work\nA number of papers have recently studied hypothesis testing problems under dierential\nprivacy guarantees [39, 40, 41]. Some works analyze the distribution of the test statistic in\nthe asymptotic regime. The work most closely related to ours is [34], which studied identity\ntesting in the \ufb01nite sample regime. We mentioned their guarantees along with our results on\nidentity testing in the previous section.\nThere has been a line of research for statistical testing and estimation problems under the\nnotion of local dierential privacy [24, 23, 42, 43, 44, 45, 46, 47, 48, 49]. These papers study\nsome basic statistical problems and provide minimax lower bounds using Fano\u2019s inequality. [50]\nstudies structured distribution estimation under dierential privacy. Information theoretic\napproaches to data privacy have been studied recently using quantities like mutual information,\nand guessing probability to quantify privacy [51, 52, 53, 54, 55].\n[56, 57] provide methods to prove lower bounds on DP algorithms via packing. Recently, [58]\nuse coupling to prove lower bounds on the sample complexity for dierentially private\ncon\ufb01dence intervals. Our results are more general, in that, we can handle mixtures of\ndistributions, which can provide optimal lower bounds on identity testing. [59, 60] characterize\n\n3\n\n\fdierential privacy through a coupling argument. [61] also uses the idea of coupling implicitly\nwhen designing dierentially private partition algorithms. [62] uses our coupling argument\nto prove lower bounds for dierentially private property estimation problems.\nIn a contemporaneous and independent work, [63], the authors study the same problems that\nwe consider, and obtain the same upper bounds for the sparse case, when m \u00c6 k. They also\nprovide experimental results to show the performance of the privatized algorithms. However,\ntheir results are sub-optimal for m =( k) for identity testing, and they do not provide any\nlower bounds for the problems. Both [34], and [63] consider only pure-dierential privacy,\nwhich are a special case of our results.\nOrganization of the paper. In Section 2, we discuss the de\ufb01nitions and notations. A\ngeneral technique for proving lower bounds for dierentially private algorithms is described\nin Section 3. Section 4 gives upper and lower bounds for identity testing, and closeness\ntesting is studied in Section 5.\n\n2\u00cep \u2260 q\u00ce1.\n\n2 Preliminaries\nLet k be the class of all discrete distributions over a domain of size k, which wlog is assumed\nto be [k] := {1, . . . ,k}. We denote length-m samples X1, . . . ,Xm by X m1 . For x \u0153 [k], let\npx be the probability of x under p. Let Mx(X m1 ) be the number of times x appears in\nX m1 . For A \u2122 [k], let p(A) = qx\u0153A px. Let X \u2265 p denote that the random variable X\nhas distribution p. Let u[k] be the uniform distribution over [k], and B(b) be the Bernoulli\ndistribution with bias b. The total variation distance between distributions p, and q over [k]\nis dT V (p, q) := supA\u00b5[k]{p(A) \u2260 q(A)} = 1\nDe\ufb01nition 1. Let p, and q be distributions over X, and Y respectively. A coupling between\np and q is a distribution over X\u25caY whose marginals are p and q respectively.\nDe\ufb01nition 2. The Hamming distance between two sequences X m1 and Y m1\nis dH(X m1 , Y m1 ) :=\nqm\ni=1 I{Xi \u201d= Yi}, the number of positions where X m1 , and Y m1 dier.\nDe\ufb01nition 3. A randomized algorithm A on a set X m \u00e6S is said to be (\u00c1, \u201d)-dierentially\nprivate if for any S \u00b5 range(A), and all pairs of X m1 , and Y m1 with dH(X m1 , Y m1 ) \u00c6 1 such\nthat Pr (A(X m1 ) \u0153 S) \u00c6 e\u00c1 \u00b7 Pr (A(Y m1 ) \u0153 S) + \u201d.\nThe case when \u201d = 0 is called pure dierential privacy. For simplicity, we denote pure\ndierential privacy as \u00c1-dierential privacy (\u00c1-DP).\nNext we state the group property of dierential privacy. We give a proof in Appendix A.1.\nLemma 1. Let A be a (\u00c1, \u201d)-DP algorithm,\nthen for sequences xm1 , and ym1 with\ndH(xm1 , ym1 ) \u00c6 t, and \u2019S \u00b5 range(A), Pr (A(xm1 ) \u0153 S) \u00c6 et\u00c1 \u00b7 Pr (A(ym1 ) \u0153 S) + \u201dte\u00c1(t\u22601).\nThe next two lemmas state a relationship between (\u00c1, \u201d) and \u00c1-dierential privacy. We give\na proof of Lemma 2 in Appendix A.2. And Lemma 3 follows from [34].\nLemma 2. Any (\u00c1 + \u201d, 0)- dierentially private algorithm is also (\u00c1, \u201d)-dierentially private.\nLemma 3. An (\u00c1, \u201d)-DP algorithm for a testing problem can be converted to an (\u00c1 + c\u201d, 0)\nalgorithm for some constant c > 0.\nCombining these two results, it suces to prove bounds for (\u00c1, 0)-DP, and plug in \u00c1 with\n(\u00c1 + \u201d) to obtain bounds that are tight up to constant factors for (\u00c1, \u201d)-DP.\nThe notion of sensitivity is useful in establishing bounds under dierential privacy.\nDe\ufb01nition 4. The sensitivity of f : [k]m \u00e6 R is\n\n(f) := maxdH(Xm\n\n1 ,Y m\n\n1 )\u00c61 |f(X m1 ) \u2260 f(Y m1 )| .\n\nFor x \u0153 R, \u2021(x) :=\nfollow from the de\ufb01nition of \u2021.\n\n1+exp(\u2260x) = exp(x)\n\n1\n\n1+exp(x) is the sigmoid function. The following properties\n\n4\n\n\fLemma 4.\n\n2. Let 0 <\u00f7< 1\n\n1. For all x, \u201c \u0153 R, exp(\u2260| \u201c|) \u00c6 \u2021(x+\u201c)\n\n\u2021(x) \u00c6 exp(|\u201c|).\n\n2. Suppose x \u00d8 log 1\n\n\u00f7. Then \u2021(x) > 1 \u2260 \u00f7.\n\nIdentity Testing (IT). Given description of q \u0153 k over [k], parameters \u2013, and m\nindependent samples X m1 from unknown p \u0153 k. A is an (k, \u2013)-identity testing algorithm for\nq, if when p = q, A outputs \u201cp = q\u201d with probability at least 0.9, and when dT V (p, q) \u00d8 \u2013,\nA outputs \u201cp \u201d= q\u201d with probability at least 0.9.\nDe\ufb01nition 5. The sample complexity of DP-identity testing, denoted S(IT, k,\u2013,\u00c1 ), is the\nsmallest m for which there exists an \u00c1-DP algorithm A that uses m samples to achieve\n(k, \u2013)-identity testing. Without privacy concerns, S(IT, k,\u2013 ) denotes the sample complexity.\nWhen q = u[k], the problem reduces to uniformity testing, and the sample complexity is\ndenoted as S(UT, k,\u2013,\u00c1 ).\nCloseness Testing (CT). Given m independent samples X m1 , and Y m1\nfrom unknown\ndistributions p, and q. An algorithm A is an (k, \u2013)-closeness testing algorithm if when p = q,\nA outputs p = q with probability at least 0.9, and when dT V (p, q) \u00d8 \u2013, A outputs p \u201d= q\nwith probability at least 0.9.\nDe\ufb01nition 6. The sample complexity of DP-closeness testing, denoted S(CT, k,\u2013,\u00c1 ), is the\nsmallest m for which there exists an \u00c1-DP algorithm A that uses m samples to achieve\n(k, \u2013)-closeness testing. When privacy is not a concern, we denote the sample complexity of\ncloseness testing as S(CT, k,\u2013 ).\nHypothesis Testing (HT). Suppose we have distributions p and q over X m, and X m1 \u2265\np, Y m1 \u2265 q, we say an algorithm A : X m \u00e6{ p, q} can distinguish between p and q if\nPr (A(X m1 ) = q) < 0.1 and Pr (A(Y m1 ) = p) < 0.1.\n3 Privacy Bounds Via Coupling\nRecall that coupling between distributions p and q over X, and Y, is a distribution over\nX\u25caY whose marginal distributions are p and q (De\ufb01nition 1). For simplicity, we treat\ncoupling as a randomized function f : X\u00e6Y such that if X \u2265 p, then Y = f(X) \u2265 q. Note\nthat X, and Y are not necessarily independent.\nExample 1. Let B(b1), and B(b2) be Bernoulli distributions with bias b1, and b2 such that\nb1 < b2. Let p, and q be distributions over {0, 1}m obtained by m i.i.d. samples from B(b1),\nand B(b2) respectively. Let X m1 be distributed according to p. Generate a sequence Y m1 as\nfollows: If Xi = 1, then Yi = 1. If Xi = 0, we \ufb02ip another coin with bias (b2\u2260b1)/(1\u2260b1), and\nlet Yi be the output of this coin. Repeat the process independently for each i, such that the\nYi\u2019s are all independent of each other. Then Pr (Yi = 1) = b1 +(1\u2260 b1)(b2 \u2260 b1)/(1\u2260 b1) = b2,\nand Y m1\nWe would like to use coupling to prove lower bounds on dierentially private algorithms for\ntesting problems. Let p and q be distributions over X m. If there is a coupling between p\nand q with a small expected Hamming distance, we might expect that the algorithm cannot\nhave strong privacy guarantees. The following theorem formalizes this intuition:\nTheorem 1. Suppose there is a coupling between p and q over X m, such that\nE [dH(X m1 , Y m1 )] \u00c6 D where X m1 \u2265 p, Y m1 \u2265 q. Then, any (\u00c1, \u201d)-dierentially private\nhypothesis testing algorithm A : X m \u00e6{ p, q} on p and q must satisfy \u00c1 + \u201d =! 1\nD\"\nProof. Let (X m1 , Y m1 ) be distributed according to a coupling of p, and q with\nE [dH(X m1 , Y m1 )] \u00c6 D.\ninequality, Pr (dH(X m1 , Y m1 ) > 10D) <\nPr (dH(X m1 , Y m1 ) > 10 \u00b7 E [dH(X m1 , Y m1 )]) < 0.1. Let xm1 and ym1 be the realization of X m1\nand Y m1 . Let W = {(xm1 , ym1 )|dH(xm1 , ym1 ) \u00c6 10D}. Then we have\n1 = ym\n\n1 ) \u00b7 Pr (A(xm\nBy Lemma 1, and Pr (dH(X m1 , Y m1 ) > 10D) < 0.1, and Pr (A(ym1 ) = q) \u00c6 1,\n\n1 ) = q) \u00d8 \u00ff(xm\n\nis distributed according to q.\n\n0.1 \u00d8 Pr (A(X m\n\nPr (X m\n\n1 = xm\n\n1 , Y m\n\nBy Markov\u2019s\n\n1 ) = q).\n\n1 ,ym\n\n1 )\u0153W\n\n5\n\n\f1 ) = q) + \u00ff(xm\n\n1 ) \u00b7 Pr (A(ym\n1 ) \u00b7 (e\u00c1\u00b710D Pr (A(xm\n\n1 ,ym\n\n1 ) /\u0153W\n\n1 ) = q) + 10D\u201d \u00b7 e\u00c1\u00b710(D\u22601)) + 0.1\n\nPr (xm\n\n1 , ym\n\n1 ) \u00b7 1\n\nPr (A(Y m\n\n1 ,ym\n\n1 ,ym\n\n1 )\u0153W\n\n1 , ym\n\n1 , ym\n\nPr (xm\n\nPr (xm\n\n1 ) = q) \u00c6 \u00ff(xm\n\u00c6 \u00ff(xm\n\u00c6 0.1e\u00c1\u00b710D + 10D\u201d \u00b7 e\u00c1\u00b710D + 0.1.\n\u00c1+\u201d2, proving the theorem.\n\n1 )\u0153W\n\n\u00c1 , 1\n\nSince we know Pr (A(Y m1 ) = q) > 0.9, then 0.9 < Pr (A(Y m1 ) = q) < 0.1e\u00c1\u00b710D + 10D\u201d \u00b7\ne\u00c1\u00b710D + 0.1. Hence, either e\u00c1\u00b710D = (1) or 10D\u201d = (1), which implies that D =\n!min) 1\n\n\u201d*\" =1 1\n\nSet \u201d = 0, we obtain the bound for pure dierential privacy. In the next few sections, we use\nthis theorem to get sample complexity bounds for dierentially private testing problems.\n\n4 Identity Testing\nIn this section, we prove the bounds for identity testing. Our main result is the following.\nTheorem 2.\n\nOr we can write it according to the parameter range,\n\n\u2013\u00c11/2 ,\n\nk1/3\n\u20134/3\u00c12/3 , 1\n\n\u20132 + max\u00d3 k1/2\n\n\u2013\u00c1\u00d42.\nS(IT, k,\u2013,\u00c1 ) =1 k1/2\n\u2013\u00c11/22,\n\u20134\" and k =! 1\nwhen k =! 1\n\u20132\u00c1\",\n\u20134/3\u00c12/32, when k =! \u2013\n\u00c1\" and k = O! 1\n\u2013\u00c12,\nwhen k = O! \u2013\n\u00c1\".\n\n\u20132 + k1/2\n\u20132 + k1/3\n\u20132 + 1\n\n1\u00d4k\n1\u00d4k\n1\u00d4k\n\n\u20134 + 1\n\n\u20132\u00c1\",\n\nS(IT, k,\u2013,\u00c1 ) =Y___]___[\n\nOur bounds are tight up to constant factors in all parameters. To get the sample complexity\nfor (\u00c1, \u201d)-dierential privacy, we can simply replace \u00c1 by (\u00c1 + \u201d).\nIn Theorem 3 we will show a reduction from identity to uniformity testing under pure\ndierential privacy. Using this, it will be enough to design algorithms for uniformity testing,\nwhich is done in Section 4.2.\nMoreover since uniformity testing is a special case of identity testing, any lower bound for\nuniformity will port over to identity, and we give such bounds in Section 4.3.\n\n4.1 Uniformity Testing implies Identity Testing\nThe sample complexity of testing identity of any distribution is O(\u00d4k\n\u20132 ), a bound that is\ntight for the uniform distribution. Recently [35] proposed a scheme to reduce the problem\nof testing identity of distributions over [k] for total variation distance \u2013 to the problem of\ntesting uniformity over [6k] with total variation parameter \u2013/3. In other words, they show\nthat S(IT, k,\u2013 ) \u00c6 S(UT, 6k, \u2013/3). Building on [35], we prove that a similar bound also holds\nfor dierentially private algorithms. The proof is in Appendix B.\nTheorem 3. S(IT, k,\u2013,\u00c1 ) \u00c6 S(UT, 6k, \u2013/3,\u00c1 ).\nIdentity Testing \u2013 Upper Bounds\n4.2\nIn this section, we will show that by privatizing the statistic proposed in [38] we can achieve\nthe sample complexity in Theorem 2 for all parameter ranges. The procedure is described in\nAlgorithm 1.\n\n6\n\n\fRecall that Mx(X m1 ) is the number of appearances of x in X m1 . Let\n\nS(X m\n\n1 ) := 1\n2 \u00b7\n\nMx(X m1 )\n\nm\n\n\u2260\n\nn\u00ffx=1----\n\n1\n\nk---- ,\n\nbe the TV distance from the empirical distribution to the uniform distribution. Let \u00b5(p) =\nE [S(X m1 )] when the samples are drawn from distribution p. They show the following\nseparation result on the expected value of S(X m1 ).\nLemma 5 ([38]). Let p be a distribution over [k] and dT V (p, u[k]) \u00d8 \u2013, then there is a\nconstant c such that\n\n(1)\n\n(2)\n\n\u00b5(p) \u2260 \u00b5(u[k]) \u00d8 c\u20132 min\u00d3 m2\n\nk , 1\n\nk2 ,\uf8ff m\n\n\u2013\u00d4.\n\n[38] used this result to show that thresholding S(X m1 ) at 0 is an optimal algorithm for\nidentity testing. We \ufb01rst normalize the statistic to simplify the presentation of our DP\nalgorithm. Let\n\nZ(X m\n\n1 ) :=Y_]_[\n\nk1S(X m1 ) \u2260 \u00b5(u[k]) \u2260 1\nm!S(X m1 ) \u2260 \u00b5(u[k]) \u2260 1\nm!S(X m1 ) \u2260 \u00b5(u[k]) \u2260 1\n\nk2 2,\n2 c\u20132 \u00b7 m2\nwhen m \u00c6 k,\nk \", when k < m \u00c6 k\n2 c\u20132 \u00b7\uf8ff m\n2 c\u2013\",\nwhen m \u00d8 k\n\u20132 .\n\n\u20132 ,\n\nwhere c is the constant in Lemma 5, and \u00b5(u[k]) is the expected value of S(X m1 ) when X m1\nare drawn from uniform distribution.\nAlgorithm 1 Uniformity testing\n\nInput: \u00c1, \u2013, i.i.d. samples X m1 from p\n\n1: Let Z(X m1 ) be evaluated from (1), and (2).\n2: Generate Y \u2265 B(\u2021(\u00c1 \u00b7 Z)), \u2021 is the sigmoid function.\n3:\n\nif Y = 0, return p = u[k], else, return p \u201d= u[k].\n\nm\n\n\u2260 1\n\nk---. Changing any one symbol changes\nk2 = 0(Xm\n\nWe now prove that this algorithm is \u00c1-DP. We need the following sensitivity result.\nLemma 6. (Z) \u00c6 1 for all values of m, and k.\nx=1--- Mx(Xm\n2 \u00b7qn\n1 )\nProof. Recall that S(X m1 ) = 1\nat most two of the Mx(X m1 )\u2019s. Therefore at most two of the terms change by at most\nm. Therefore, (S(X m1 )) \u00c6 1\n1\nm, for any m. When m \u00c6 k, this can be strengthened\nwith observation that Mx(X m1 )/m \u00d8 1\nk, for all Mx(X m1 ) \u00d8 1. Therefore, S(X m1 ) = 1\n2 \u00b7\nk2 +qx:Mx(Xm\n1qx:Mx(Xm\n\u2260 1\n, where 0(X m1 ) is the number\nof symbols not appearing in X m1 . This changes by at most one when one symbol is changed,\nproving the result.\nUsing this lemma, \u00c1 \u00b7 Z(X m1 ) changes by at most \u00c1 when X m1\nis changed at one location.\nInvoking Lemma 4, the probability of any output changes by a multiplicative exp(\u00c1), and\nthe algorithm is \u00c1-dierentially private.\nTo prove the sample complexity bound, we \ufb01rst show that the mean of the test statistic is\nwell separated using Lemma 5. Then we use the concentration bound of the test statistic\nfrom [38] to get the \ufb01nal complexity. Due to lack of space, the detailed proof of sample\ncomplexity bound is given in Appendix C.\n\n1 )\u00d811 Mx(Xm\n\n1 )=0\n\n1 )\n\n1 )\n\nm\n\n1\n\nk\n\n4.3 Sample Complexity Lower bounds for Uniformity Testing\nIn this section, we will show the lower bound part of Theorem 2. The \ufb01rst term is the lower\nbound without privacy constraints, proved in [7]. In this section, we will prove the terms\nassociated with privacy.\n\n7\n\n\f\u2013\u00c1).\n\nThe simplest argument is for m \u00d8 k\n\u20132 , which hopefully will give you a sense of how coupling\nargument works. We consider the case of binary identity testing where the goal is to test\nwhether the bias of a coin is 1/2 or \u2013-far from 1/2. This is a special case of identity testing\nfor distributions over [k] (when k \u2260 2 symbols have probability zero). This is strictly harder\nthan the problem of distinguishing between B(1/2) and B(1/2 + \u2013). The coupling given in\nExample 1 has expected hamming distance of \u2013m. Hence combing with Theorem 1, we get\na lower bound of ( 1\nWe now consider the cases m \u00c6 k and k < m \u00c6 k\n\u20132 .\nTo this end, we invoke LeCam\u2019s two point theorem, and design a hypothesis testing problem\nthat will imply a lower bound on uniformity testing. The testing problem will be to distinguish\nbetween the following two cases.\nCase 1: We are given m independent samples from the uniform distribution u[k].\nCase 2: Generate a distribution p with dT V (p, u[k]) \u00d8 \u2013 according to some prior over all\nsuch distributions. We are then given m independent samples from this distribution p.\nLe Cam\u2019s two point theorem [64] states that any lower bound for distinguishing between\nthese two cases is a lower bound on identity testing problem.\nWe now describe the prior construction for Case 2, which is the same as considered by [7]\nfor lower bounds on identity testing without privacy considerations. For each z \u0153 {\u00b11}k/2,\nde\ufb01ne a distribution pz over [k] such that\npz(2i \u2260 1) = 1 + zi \u00b7 2\u2013\n\n, and pz(2i) = 1 \u2260 zi \u00b7 2\u2013\n\nk\n\n.\n\nk\n\nThen for any z, dT V (Pz, u[k]) = \u2013. For Case 2, choose p uniformly from these 2k/2\ndistributions. Let Q2 denote the distribution on [k]m by this process. In other words, Q2 is\na mixture of product distributions over [k].\nIn Case 1, let Q1 be the distribution of m i.i.d. samples from u[k].\nTo obtain a sample complexity lower bound for distinguishing the two cases, we will design\na coupling between Q1, and Q2, and bound its expected Hamming distance. While it can be\nshown that the Hamming distance of the coupling between the uniform distribution with\nany one of the 2k/2 distributions grows as \u2013m, it can be signi\ufb01cantly smaller, when we\nconsider the mixtures. In particular, the following lemma shows that there exist couplings\nwith bounded Hamming distance.\nLemma 7. There is a coupling between X m1 generated by Q1, and Y m1\n\nby Q2 such that\n\nE [dH(X m1 , Y m1 )] \u00c6 C \u00b7 \u20132 min{ m2\n\nk , m3/2\nk1/2 }.\n\nThe lemma is proved in Appendix D. Now applying Theorem 1, we get the bound in\nTheorem 2.\n\n5 Closeness Testing\nRecall the closeness testing problem from Section 2, and the tight non-private bounds from\nTable 1. Our main result in this section is the following theorem characterizing the sample\ncomplexity of dierentially private algorithms for closeness testing.\nTheorem 4. If \u2013> 1/k1/4, and \u00c1\u20132 > 1/k,\n\notherwise,\n\nS(CT, k,\u2013,\u00c1 ) =3 k2/3\n\u2013\u00d4\u00c14,\n\u20134/3 + k1/2\n\u2013\u00c14 \u00c6 S(CT, k,\u2013,\u00c1 ) \u00c6 O3 k1/2\n+ 1\n\n\u20132\u00c14.\n\u20132 + 1\n\n3 k1/2\n\n\u20132 + k1/2\n\u2013\u00d4\u00c1\n\n8\n\n\fThis theorem shows that in the sparse regime, when m = O(k), our bounds are tight up to\nconstant factors in all parameters. To prove the upper bounds, we only consider the case\nwhen \u201d = 0, which would suce by lemma 2. We privatize the closeness testing algorithm\nof [10]. To reduce the strain on the readers, we drop the sequence notations explicitly and let\n\nThe statistic used by [10] is\n\n\u00b5i := Mi(X m\n\n1 ), and \u2039i := Mi(Y m\n1 ).\n\nZ(X m\n\n1 , Y m\n\n1 ) := \u00ffi\u0153[k]\n\n(\u00b5i \u2260 \u2039i)2 \u2260 \u00b5i \u2260 \u2039i\n\n\u00b5i + \u2039i\n\n,\n\nwhere we assume that ((\u00b5i \u2260 \u2039i)2 \u2260 \u00b5i \u2260 \u2039i)/(\u00b5i + \u2039i) = 0, when \u00b5i + \u2039i = 0. It turns out\nthat this statistic has a constant sensitivity, as shown in Lemma 8.\nLemma 8. (Z(X m1 , Y m1 )) \u00c6 14.\nProof. Since Z(X m1 , Y m1 ) is symmetric, without loss of generality assume that one of the\nsymbols is changed in Y m1 . This would cause at most two of the \u2039i\u2019s to change. Suppose\n\u2039i \u00d8 1, and it changed to \u2039i \u2260 1. Suppose, \u00b5i + \u2039i > 1, the absolute change in the ith term\nof the statistic is\n\n(\u00b5i \u2260 \u2039i)2\n\u00b5i + \u2039i \u2260\n\n----\n\n(\u00b5i \u2260 \u2039i + 1)2\n\n\u00b5i + \u2039i \u2260 1 ---- =----\n\u00c6----\n\n\u00c6\n\n(\u00b5i + \u2039i)(2\u00b5i \u2260 2\u2039i + 1) + (\u00b5i \u2260 \u2039i)2\n\u00b5i + \u2039i \u2260 1 ---- +----\n2\u00b5i \u2260 2\u2039i + 1\n3|\u00b5i \u2260 \u2039i| + 1\n\u00b5i + \u2039i \u2260 1 \u00c6 3 +\n\n(\u00b5i + \u2039i)(\u00b5i + \u2039i \u2260 1)\n\u00b5i + \u2039i \u2260 1----\n\u00b5i \u2260 \u2039i\n4\n\u00b5i + \u2039i \u2260 1 \u00c6 7.\n\n----\n\nWhen \u00b5i + \u2039i = 1, the change can again be bounded by 7. Since at most two of the \u2039i\u2019s\nchange, we obtain the desired bound.\n\nWe use the same approach with the test statistic as with uniformity testing to obtain a\ndierentially private closeness testing method, described in Algorithm 2. Since the sensitivity\nof the statistic is at most 14, the input to the sigmoid changes by at most \u00c1 when any\ninput sample is changed. Invoking Lemma 4, the probability of any output changes by a\nmultiplicative exp(\u00c1), and the algorithm is \u00c1-dierentially private.\n\nAlgorithm 2\n\nInput: \u00c1, \u2013, sample access to distribution p and q\n\nm2\u20132\n4k+2m)/14\n\n1: Z\u00d5 \u03a9 (Z(X m1 , Y m1 ) \u2260 1\n2\n2: Generate Y \u2265 B(\u2021(exp(\u00c1 \u00b7 Z\u00d5))\nif Y = 0, return p = q\n3:\n4: else, return p \u201d= q\n\nThe remaining part is to show that Algorithm 2 satis\ufb01es sample complexity upper bounds\ndescribed in theorem 4. We will give the details in Appendix E, where the analysis of the\nlower bound is also given.\n\nAcknowledgement\nThe authors thank Gautam Kamath for some very helpful suggestions about this work.\n\n9\n\n\fReferences\n[1] Jerzy Neyman and Egon Sharpe Pearson. On the problem of the most ecient tests of\nstatistical hypotheses. Philosophical Transactions of the Royal Society of London. Series\nA, Containing Papers of a Mathematical or Physical Character, 231:289\u2013337, 1933.\n\n[2] Erich Leo Lehmann and George Casella. Theory of Point Estimation, volume 31.\n\nSpringer, 2006.\n\n[3] Tukan Batu, Lance Fortnow, Ronitt Rubinfeld, Warren D. Smith, and Patrick White.\nTesting that distributions are close. In Proceedings of the 41st Annual IEEE Symposium\non Foundations of Computer Science, FOCS \u201900, pages 259\u2013269, Washington, DC, USA,\n2000. IEEE Computer Society.\n\n[4] Tukan Batu, Eldar Fischer, Lance Fortnow, Ravi Kumar, Ronitt Rubinfeld, and Patrick\nWhite. Testing random variables for independence and identity. In Proceedings of the\n42nd Annual IEEE Symposium on Foundations of Computer Science, FOCS \u201901, pages\n442\u2013451, Washington, DC, USA, 2001. IEEE Computer Society.\n\n[5] Oded Goldreich and Dana Ron. On testing expansion in bounded-degree graphs.\nIn Studies in Complexity and Cryptography. Miscellanea on the Interplay between\nRandomness and Computation, pages 68\u201375. Springer, 2011.\n\n[6] Tugkan Batu. Testing properties of distributions. PhD thesis, Cornell University, 2001.\n[7] Liam Paninski. A coincidence-based test for uniformity given very sparsely sampled\n\ndiscrete data. IEEE Transactions on Information Theory, 54(10):4750\u20134755, 2008.\n\n[8] Jayadev Acharya, Ashkan Jafarpour, Alon Orlitsky, and Ananda Theertha Suresh. A\ncompetitive test for uniformity of monotone distributions. In Proceedings of the 16th\nInternational Conference on Arti\ufb01cial Intelligence and Statistics, 2013.\n\n[9] Jayadev Acharya, Ashkan Jafarpour, Alon Orlitksy, and Ananda Theertha Suresh. Sub-\nlinear algorithms for outlier detection and generalized closeness testing. In Proceedings\nof the 2014 IEEE International Symposium on Information Theory, 2014.\n\n[10] Siu-On Chan, Ilias Diakonikolas, Gregory Valiant, and Paul Valiant. Optimal algorithms\nfor testing closeness of discrete distributions. In Proceedings of the 25th Annual ACM-\nSIAM Symposium on Discrete Algorithms, SODA \u201914, pages 1193\u20131203, Philadelphia,\nPA, USA, 2014. SIAM.\n\n[11] Ilias Diakonikolas, Daniel M. Kane, and Vladimir Nikishkin. Testing identity of struc-\ntured distributions. In Proceedings of the 26th Annual ACM-SIAM Symposium on\nDiscrete Algorithms, SODA \u201915, pages 1841\u20131854, Philadelphia, PA, USA, 2015. SIAM.\n[12] Bhaswar Bhattacharya and Gregory Valiant. Testing closeness with unequal sized\nIn Advances in Neural Information Processing Systems, NIPS \u201915, pages\n\nsamples.\n2611\u20132619. Curran Associates, Inc., 2015.\n\n[13] Cl\u00e9ment L. Canonne, Ilias Diakonikolas, Themis Gouleakis, and Ronitt Rubinfeld.\nTesting shape restrictions of discrete distributions. In Proceedings of the 33rd Symposium\non Theoretical Aspects of Computer Science, STACS \u201916, pages 25:1\u201325:14, Dagstuhl,\nGermany, 2016. Schloss Dagstuhl\u2013Leibniz-Zentrum fuer Informatik.\n\n[14] Ilias Diakonikolas and Daniel M. Kane. A new approach for testing properties of discrete\ndistributions. In Proceedings of the 57th Annual IEEE Symposium on Foundations\nof Computer Science, FOCS \u201916, pages 685\u2013694, Washington, DC, USA, 2016. IEEE\nComputer Society.\n\n[15] Tukan Batu and Cl\u00e9ment L. Canonne. Generalized uniformity testing. In Proceedings\nof the 58th Annual IEEE Symposium on Foundations of Computer Science, FOCS \u201917,\npages 880\u2013889, Washington, DC, USA, 2017. IEEE Computer Society.\n\n10\n\n\f[16] Cl\u00e9ment L. Canonne. A survey on distribution testing: Your data is big. but is it blue?\n\nElectronic Colloquium on Computational Complexity (ECCC), 22(63):63, 2015.\n\n[17] Arvind Narayanan and Vitaly Shmatikov. Robust de-anonymization of large sparse\ndatasets. In Proceesings of the 29th IEEE Symposium on Security and Privacy, pages\n111\u2013125, 2008.\n\n[18] Latanya Sweeney. k-anonymity: A model for protecting privacy. International Journal\n\nof Uncertainty, Fuzziness and Knowledge-Based Systems, 10(05):557\u2013570, 2002.\n\n[19] Stanley L Warner. Randomized response: A survey technique for eliminating evasive\n\nanswer bias. Journal of the American Statistical Association, 60(309):63\u201369, 1965.\n\n[20] Tore Dalenius. Towards a methodology for statistical disclosure control. Statistisk\n\nTidskrift, 15:429\u2013444, 1977.\n\n[21] Irit Dinur and Kobbi Nissim. Revealing information while preserving privacy.\n\nIn\nProceedings of the 22nd ACM SIGMOD-SIGACT-SIGART Symposium on Principles of\nDatabase Systems, PODS \u201903, pages 202\u2013210, New York, NY, USA, 2003. ACM.\n\n[22] Larry Wasserman and Shuheng Zhou. A statistical framework for dierential privacy.\n\nJournal of the American Statistical Association, 105(489):375\u2013389, 2010.\n\n[23] John C Duchi, Michael I Jordan, and Martin J Wainwright. Local privacy and statistical\nminimax rates. In Proceedings of the 54st Annual IEEE Symposium on Foundations of\nComputer Science, FOCS \u201913, pages 429\u2013438. IEEE, 2013.\n\n[24] Martin J Wainwright, Michael I Jordan, and John C Duchi. Privacy aware learning. In\n\nAdvances in Neural Information Processing Systems, pages 1430\u20131438, 2012.\n\n[25] Kamalika Chaudhuri, Claire Monteleoni, and Anand D Sarwate. Dierentially private\nempirical risk minimization. Journal of Machine Learning Research, 12:1069\u20131109, 2011.\n[26] Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to\nsensitivity in private data analysis. In Proceedings of the 3rd Conference on Theory of\nCryptography, TCC \u201906, pages 265\u2013284, Berlin, Heidelberg, 2006. Springer.\n\n[27] Cynthia Dwork. Dierential privacy: A survey of results. In Proceedings of the 5th\nInternational Conference on Theory and Applications of Models of Computation, TAMC\n\u201908, pages 1\u201319, Berlin, Heidelberg, 2008. Springer.\n\n[28] Cynthia Dwork, Guy N. Rothblum, and Salil Vadhan. Boosting and dierential privacy.\nIn Proceedings of the 51st Annual IEEE Symposium on Foundations of Computer Science,\nFOCS \u201910, pages 51\u201360, Washington, DC, USA, 2010. IEEE Computer Society.\n\n[29] Avrim Blum, Katrina Ligett, and Aaron Roth. A learning theory approach to noninter-\n\nactive database privacy. Journal of the ACM (JACM), 60(2):12, 2013.\n\n[30] Frank McSherry and Kunal Talwar. Mechanism design via dierential privacy. In 48th\nAnnual IEEE Symposium on Foundations of Computer Science, pages 94\u2013103. IEEE,\n2007.\n\n[31] Chao Li, Gerome Miklau, Michael Hay, Andrew McGregor, and Vibhor Rastogi. The\nmatrix mechanism: Optimizing linear counting queries under dierential privacy. The\nVLDB Journal, 24(6):757\u2013781, 2015.\n\n[32] Peter Kairouz, Sewoong Oh, and Pramod Viswanath. The composition theorem for\ndierential privacy. IEEE Transactions on Information Theory, 63(6):4037\u20134049, 2017.\n[33] Cynthia Dwork and Aaron Roth. The algorithmic foundations of dierential privacy.\n\nFoundations and Trends R\u2022 in Theoretical Computer Science, 9(3\u20134):211\u2013407, 2014.\n[34] Bryan Cai, Constantinos Daskalakis, and Gautam Kamath. Priv\u2019it: Private and sample\necient identity testing. In Proceedings of the 34th International Conference on Machine\nLearning, ICML \u201917, pages 635\u2013644. JMLR, Inc., 2017.\n\n11\n\n\f[35] Oded Goldreich. The uniform distribution is complete with respect to testing identity\nto a \ufb01xed distribution. In Electronic Colloquium on Computational Complexity (ECCC),\nvolume 23, 2016.\n\n[36] Gregory Valiant and Paul Valiant. An automatic inequality prover and instance optimal\nidentity testing. In Proceedings of the 55th Annual IEEE Symposium on Foundations of\nComputer Science, pages 51\u201360. IEEE, 2014.\n\n[37] Jayadev Acharya, Constantinos Daskalakis, and Gautam C Kamath. Optimal testing\nfor properties of distributions. In Advances in Neural Information Processing Systems,\nNIPS \u201915, pages 3577\u20133598. Curran Associates, Inc., 2015.\n\n[38] Ilias Diakonikolas, Themis Gouleakis, John Peebles, and Eric Price. Sample-optimal\nidentity testing with high probability. In Proceedings of the 45th International Colloquium\non Automata, Languages, and Programming, ICALP \u201918, pages 41:1\u201341:14, 2018.\n\n[39] Yue Wang, Jaewoo Lee, and Daniel Kifer. Revisiting dierentially private hypothesis\n\ntests for categorical data. arXiv preprint arXiv:1511.03376, 2015.\n\n[40] Marco Gaboardi, Hyun-Woo Lim, Ryan M. Rogers, and Salil P. Vadhan. Dierentially\nprivate chi-squared hypothesis testing: Goodness of \ufb01t and independence testing. In\nProceedings of the 33rd International Conference on Machine Learning, ICML \u201916, pages\n1395\u20131403. JMLR, Inc., 2016.\n\n[41] Ryan Rogers and Daniel Kifer. A New Class of Private Chi-Square Hypothesis Tests.\nIn Aarti Singh and Jerry Zhu, editors, Proceedings of the 20th International Conference\non Arti\ufb01cial Intelligence and Statistics, volume 54 of Proceedings of Machine Learning\nResearch, pages 991\u20131000, Fort Lauderdale, FL, USA, 20\u201322 Apr 2017. PMLR.\n\n[42] \u00dalfar Erlingsson, Vasyl Pihur, and Aleksandra Korolova. RAPPOR: Randomized\naggregatable privacy-preserving ordinal response. In Proceedings of the 2014 ACM\nConference on Computer and Communications Security, CCS \u201914, pages 1054\u20131067,\nNew York, NY, USA, 2014. ACM.\n\n[43] Adriano Pastore and Michael Gastpar. Locally dierentially-private distribution estima-\ntion. In Proceedings of the 2016 IEEE International Symposium on Information Theory,\npages 2694\u20132698, 2016.\n\n[44] Peter Kairouz, Keith Bonawitz, and Daniel Ramage. Discrete distribution estimation\nunder local privacy. In Proceedings of the 33rd International Conference on International\nConference on Machine Learning - Volume 48, ICML\u201916, pages 2436\u20132444, 2016.\n\n[45] Shaowei Wang, Liusheng Huang, Pengzhan Wang, Yiwen Nie, Hongli Xu, Wei Yang,\nXiang-Yang Li, and Chunming Qiao. Mutual information optimally local private discrete\ndistribution estimation. arXiv preprint arXiv:1607.08025, 2016.\n\n[46] Min Ye and Alexander Barg. Optimal schemes for discrete distribution estimation under\nlocally dierential privacy. IEEE Transactions on Information Theory, 64:5662\u20135676,\n2018.\n\n[47] Jayadev Acharya, Ziteng Sun, and Huanyu Zhang. Hadamard response: Estimat-\ning distributions privately, eciently, and with little communication. arXiv preprint\narXiv:1802.04705, 2018.\n\n[48] Or Sheet. Locally private hypothesis testing. In Proceedings of the 35th International\nConference on Machine Learning, volume 80, pages 4612\u20134621. PMLR, 10\u201315 Jul 2018.\n[49] Jayadev Acharya, Cl\u00e9ment L Canonne, Cody Freitag, and Himanshu Tyagi. Test without\ntrust: Optimal locally private distribution testing. arXiv preprint arXiv:1808.02174,\n2018.\n\n[50] Ilias Diakonikolas, Moritz Hardt, and Ludwig Schmidt. Dierentially private learning of\nstructured discrete distributions. In Advances in Neural Information Processing Systems\n28, NIPS \u201915, pages 2566\u20132574. Curran Associates, Inc., 2015.\n\n12\n\n\f[51] Darakhshan J Mir. Information-theoretic foundations of dierential privacy. In Interna-\n\ntional Symposium on Foundations and Practice of Security, pages 374\u2013381, 2012.\n\n[52] Lalitha Sankar, S Raj Rajagopalan, and H Vincent Poor. Utility-privacy tradeos in\ndatabases: An information-theoretic approach. IEEE Transactions on Information\nForensics and Security, 8(6):838\u2013852, 2013.\n\n[53] Paul Cu and Lanqing Yu. Dierential privacy as a mutual information constraint. In\nACM SIGSAC Conference on Computer and Communications Security, pages 43\u201354.\nACM, 2016.\n\n[54] Weina Wang, Lei Ying, and Junshan Zhang. On the relation between identi\ufb01ability,\ndierential privacy, and mutual-information privacy. IEEE Transactions on Information\nTheory, 62(9):5018\u20135029, 2016.\n\n[55] Ibrahim Issa and Aaron B. Wagner. Operational de\ufb01nitions for some common infor-\nmation leakage metrics. In Proceedings of the 2017 IEEE International Symposium on\nInformation Theory, ISIT \u201917, 2017.\n\n[56] Moritz Hardt and Kunal Talwar. On the geometry of dierential privacy. In Proceedings\nof the Forty-Second ACM Symposium on Theory of Computing, pages 705\u2013714. ACM,\n2010.\n\n[57] Salil Vadhan. The complexity of dierential privacy. In Yehuda Lindell, editor, Tutorials\non the Foundations of Cryptography: Dedicated to Oded Goldreich, chapter 7, pages\n347\u2013450. Springer International Publishing AG, Cham, Switzerland, 2017.\n\n[58] Vishesh Karwa and Salil Vadhan. Finite sample dierentially private con\ufb01dence intervals.\nIn Proceedings of the 9th Conference on Innovations in Theoretical Computer Science,\nITCS \u201918, pages 44:1\u201344:9. Schloss Dagstuhl\u2013Leibniz-Zentrum fuer Informatik, 2018.\n\n[59] Gilles Barthe, Marco Gaboardi, Benjamin Gr\u00e9goire, Justin Hsu, and Pierre-Yves Strub.\nProving dierential privacy via probabilistic couplings. In Proceedings of the 31st Annual\nACM/IEEE Symposium on Logic in Computer Science, pages 749\u2013758. ACM, 2016.\n\n[60] Gilles Barthe, No\u00e9mie Fong, Marco Gaboardi, Benjamin Gr\u00e9goire, Justin Hsu, and Pierre-\nYves Strub. Advanced probabilistic couplings for dierential privacy. In Proceedings of\nthe 2016 ACM SIGSAC Conference on Computer and Communications Security, pages\n55\u201367. ACM, 2016.\n\n[61] Cynthia Dwork, Moni Naor, Omer Reingold, and Guy N Rothblum. Pure dierential\nprivacy for rectangle queries via private partitions. In International Conference on\nthe Theory and Application of Cryptology and Information Security, pages 735\u2013751.\nSpringer, 2015.\n\n[62] Jayadev Acharya, Gautam Kamath, Ziteng Sun, and Huanyu Zhang. INSPECTRE:\nPrivately estimating the unseen. In Jennifer Dy and Andreas Krause, editors, Proceedings\nof the 35th International Conference on Machine Learning, volume 80 of Proceedings of\nMachine Learning Research, pages 30\u201339, Stockholmsm\u00e4ssan, Stockholm Sweden, 10\u201315\nJul 2018. PMLR.\n\n[63] Maryam Aliakbarpour, Ilias Diakonikolas, and Ronitt Rubinfeld. Dierentially private\nidentity and equivalence testing of discrete distributions. In Proceedings of the 35th\nInternational Conference on Machine Learning, pages 169\u2013178, 2018.\n\n[64] Bin Yu. Assouad, Fano, and Le Cam. In Festschrift for Lucien Le Cam, pages 423\u2013435.\n\nSpringer New York, 1997.\n\n[65] Jayadev Acharya, Alon Orlitsky, Ananda Theertha Suresh, and Himanshu Tyagi. Es-\ntimating R\u00e9nyi entropy of discrete distributions. IEEE Transactions on Information\nTheory, 63(1):38\u201356, Jan 2017.\n\n[66] Andreas Knoblauch. Closed-form expressions for the moments of the binomial probability\n\ndistribution. SIAM Journal on Applied Mathematics, 69(1):197\u2013204, 2008.\n\n13\n\n\f[67] Frank den Hollander. Probability theory: The coupling method. Lecture notes available\nonline (http://websites. math. leidenuniv. nl/probability/lecturenotes/CouplingLectures.\npdf), 2012.\n\n14\n\n\f", "award": [], "sourceid": 3440, "authors": [{"given_name": "Jayadev", "family_name": "Acharya", "institution": "Cornell University"}, {"given_name": "Ziteng", "family_name": "Sun", "institution": "Cornell University"}, {"given_name": "Huanyu", "family_name": "Zhang", "institution": "Cornell University"}]}