{"title": "Facility Location Problem in Differential Privacy Model Revisited", "book": "Advances in Neural Information Processing Systems", "page_first": 8491, "page_last": 8500, "abstract": "In this paper we study the facility location problem in the model of differential privacy (DP) with uniform facility cost. Specifically, we first show that under the hierarchically well-separated tree (HST) metrics and the super-set output setting that was introduced in Gupta et. al., there is an $\\epsilon$-DP algorithm that achieves an $O(\\frac{1}{\\epsilon})$(expected multiplicative) approximation ratio; this implies an $O(\\frac{\\log n}{\\epsilon})$ approximation ratio for the general metric case, where $n$ is the size of the input metric. These bounds improve the best-known results given by Gupta et. al.  In particular, our approximation ratio for HST-metrics is independent of $n$, and the ratio for general metrics is independent of the aspect ratio of the input metric. On the negative side, we show that the approximation ratio of any $\\epsilon$-DP algorithm is lower bounded by $\\Omega(\\frac{1}{\\sqrt{\\epsilon}})$, even for instances on HST metrics with uniform facility cost, under the super-set output setting. The lower bound shows that the dependence of the approximation ratio for HST metrics on $\\epsilon$ can not be removed or greatly improved. Our novel methods and techniques for both the upper and lower bound may find additional applications.", "full_text": "Facility Location Problem in Differential Privacy\n\nModel Revisited\n\nYunus Esencayi \u2217\nSUNY at Buffalo\n\nyunusese@buffalo.edu\n\nMarco Gaboardi\nBoston University\n\ngaboardi@bu.edu\n\nShi Li\n\nSUNY at Buffalo\n\nshil@buffalo.edu\n\nDi Wang\n\nSUNY at Buffalo\n\ndwang45@buffalo.edu\n\nAbstract\n\nIn this paper we study the uncapacitated facility location problem in the model\nof differential privacy (DP) with uniform facility cost. Speci\ufb01cally, we \ufb01rst show\nthat, under the hierarchically well-separated tree (HST) metrics and the super-set\noutput setting that was introduced in [8], there is an \u0001-DP algorithm that achieves\n\u0001 ) (expected multiplicative) approximation ratio; this implies an O( log n\nan O( 1\n)\napproximation ratio for the general metric case, where n is the size of the input\nmetric. These bounds improve the best-known results given by [8]. In particular,\nour approximation ratio for HST-metrics is independent of n, and the ratio for\ngeneral metrics is independent of the aspect ratio of the input metric.\nOn the negative side, we show that the approximation ratio of any \u0001-DP algorithm\nis lower bounded by \u2126( 1\u221a\n\u0001 ), even for instances on HST metrics with uniform\nfacility cost, under the super-set output setting. The lower bound shows that the\ndependence of the approximation ratio for HST metrics on \u0001 can not be removed or\ngreatly improved. Our novel methods and techniques for both the upper and lower\nbound may \ufb01nd additional applications.\n\n\u0001\n\nIntroduction\n\n1\nThe facility location problem is one of the most fundamental problems in combinatorial optimization\nand has a wide range of applications such as plant or warehouse location problems and network design\nproblems, also it is closely related to clustering problems such as k-median, where one typically\nseeks to partition a set of data points, which themselves \ufb01nd applications in data mining, machine\nlearning, and bioinformatics [1, 13, 4]. Due to its versatility, the problem has been studied by both\noperations research and computer science communities [20, 19, 16, 15, 1, 13, 4]. Formally, it can be\nde\ufb01ned as following.\nDe\ufb01nition 1 (Uniform Facility Location Problem (Uniform-FL)). The input to the Uniform Facility\nLocation (Uniform-FL) problem is a tuple (V, d, f, (cid:126)N ), where (V, d) is a n-point discrete metric,\nf \u2208 R\u22650 is the facility cost, and (cid:126)N = (Nv)v\u2208V \u2208 ZV\u22650 gives the number of clients in each location\nv \u2208 V . The goal of the problem is to \ufb01nd a set of facility locations S \u2286 V which minimize the\nfollowing, where d(v, S) = mins\u2208S d(v, s),\n\n(cid:88)\n\nv\u2208V\n\ncostd(S; (cid:126)N ) := |S| \u00b7 f +\n\nmin\nS\u2286V\n\nNvd(v, S).\n\n(1)\n\nThe \ufb01rst term of (1) is called the facility cost and the second term is called the connection cost.\n\n\u2217Authors are alphabetically ordered.\n\n33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada.\n\n\f\u221a\n\nThroughout the paper, we shall simply use UFL to refer to Uniform-FL. Although the problem has\nbeen studied quite well in recent years, there is some privacy issue on the locations of the clients.\nConsider the following scenario: One client may get worried that the other clients may be able\nto obtain some information on her location by colluding and exchanging their information. As a\ncommonly-accepted approach for preserving privacy, Differential Privacy (DP) [5] provides provable\nprotection against identi\ufb01cation and is resilient to arbitrary auxiliary information that might be\navailable to attackers.\nHowever, under the \u0001-DP model, Gupta et al. [8] recently showed that it is impossible to achieve a\nuseful multiplicative approximation ratio of the facility location problem. Speci\ufb01cally, they showed\nthat any 1-DP algorithm for UFL under general metric that outputs the set of open facilities must\nhave a (multiplicative) approximation ratio of \u2126(\nn) which negatively shows that UFL in DP model\nis useless. Thus one needs to consider some relaxed settings in order to address the issue.\nIn the same paper [8] the authors showed that, under the following setting, an O( log2 n log2 \u2206\n)\napproximation ratio under the \u0001-DP model is possible, where \u2206 = maxu,v\u2208V d(u, v) is the diameter\nof the input metric. In the setting, the output is a set R \u2286 V , which is a super-set of the set of open\nfacilities. Then every client sees the output R and chooses to connect to its nearest facility in R. The\nthe actual set S of open facilities, is the facilities in R with at least one connecting client. Thus, in\nthis model, a client will only know its own service facility, instead of the set of open facilities.\nWe call this setting the super-set output setting. Roughly speaking, under the \u0001-DP model, one can\nnot well distinguish between if there is 0 or 1 client at some location v. If v is far away from all the\nother locations, then having one client at v will force the algorithm to open v and thus will reveal\ninformation about the existence of the client at v. This is how the lower bound in [8] was established.\nBy using the super-set output setting, the algorithm can always output v and thus does not reveal\nmuch information about the client. If there is no client at v then v will not be open.\nIn this paper we further study the UFL problem in the \u0001-DP model with the super-set output setting\nby [8] we address the following questions.\n\n\u0001\n\nFor the UFL problem under the \u0001-DP model and the super-set output setting, can\nwe do better than the results in [8] in terms of the approximation ratio? Also, what\nis the lower bound of the approximation ratio in the same setting?\n\nWe make progresses on both problems. Our contributions can be summarized as the followings.\n\u2022 We show that under the so called Hierarchical-Well-Separated-Tree (HST) metrics, there is an\n\u0001 ) approximation ratio. By using the classic FRT tree embedding\nalgorithm that achieves O( 1\ntechnique of [6], we can achieve O( log n\n) approximation ratio for any metrics, under the \u0001-DP\nmodel and the super-set output setting. These factors respectively improve upon a factor of\nO(log n log2 \u2206) in [8] for HST and general metrics. Thus, for HST-metrics, our approximation\nonly depends on \u0001. For general metrics, our result removed the poly-logarithmic dependence on \u2206\nin [8].\n\u2022 On the negative side, we show that the approximation ratio under \u0001-DP model is lower bounded\n\u0001 ) even if the metric is a star (which is a special case of a HST). This shows that the\n\nby \u2126( 1\u221a\ndependence on \u0001 is unavoidable and can not been improved greatly.\n\n\u0001\n\nRelated Work The work which is the most related to this paper is [8], where the author \ufb01rst studied\nthe problem. Nissim et al. [18] study an abstract mechanism design model where DP is used to design\napproximately optimal mechanism, and they use facility location as one of their key examples. The\nUFL problem has close connection to k-median clustering and submodular optimization, whose DP\nversions have been studied before such as [17, 3, 7, 8, 2]. However, their methods cannot be used in\nour problem. There are many papers study other combinatorial optimization problems in DP model\nsuch as [9, 10, 11, 12, 8]. Finally, we remark that the setting we considered in the paper is closely\nrelated to the Joint Differential Privacy Model that was introduced in [14]. We leave the details to the\nfull version of the paper.\n2 Preliminaries\nGiven a data universe V and a dataset D = {v1,\u00b7\u00b7\u00b7 , vN} \u2208 V N where each record vi belongs to\nan individual i whom we refer as a client in this paper. Let A : V N (cid:55)\u2192 S be an algorithm on D and\nproduce an output in S. Let D\u2212i denote the dataset D without entry of the i-th client. Also (v(cid:48)\ni, D\u2212i)\ndenote the dataset by adding v(cid:48)\n\ni to D\u2212i.\n\n2\n\n\fi, D\u2212i) \u2208 T ].\n\nDe\ufb01nition 2 (Differential Privacy [5]). A randomized algorithm A is \u0001-differentially private (DP) if\nfor any client i \u2208 [N ], any two possible data entries vi, v(cid:48)\ni \u2208 V , any dataset D\u2212i \u2208 V N\u22121 and for all\nevents T \u2286 S in the output space of A, we have Pr[A(vi, D\u2212i) \u2208 T ] \u2264 e\u0001 Pr[A(v(cid:48)\nFor the UFL problem, instead of using a set D of clients as input, it is more convenient for us to use a\nvector (cid:126)N = (Nv)v\u2208V \u2208 ZV\u22650, where Nv indicates the number of clients at location v. Then the \u0001-DP\nrequires that for any input vectors (cid:126)N and (cid:126)N(cid:48) with | (cid:126)N \u2212 (cid:126)N(cid:48)|1 = 1 and any event T \u2286 S, we have\nPr[A( (cid:126)N ) \u2208 T ] \u2264 e\u0001 Pr[A( (cid:126)N(cid:48)) \u2208 T ].\nIn the super-set output setting for the UFL problem, the output of an algorithm is a set R \u2286 V of\npotential open facilities. Then, every client, or equivalently, every location v with Nv \u2265 1, will be\nconnected to the nearest location in R under some given metric (in our algorithm, we use the HST tree\nmetric). Then the actual set S of open facilities is the set of locations in R with at least 1 connected\nclient. Notice that the connection cost of S will be the same as that of R; but the facility cost might\nbe much smaller than that of R. This is why the super-set output setting may help in getting good\napproximation ratios.\nThroughout the paper, approximation ratio of an algorithm A is the expected multiplicative\napproximation ratio, which is the expected cost of the solution given by the algorithm, divided\nby the cost of the optimum solution, i.e.,\n, where the expectation is over the\nrandomness of A.\nOrganization In Section 3, we show how to reduce UFL on general metrics to that on HST\nmetrics, while losing a factor of O(log n) in the approximation ratio. In Section 4, we give our \u0001-DP\n\u221a\nO(1/\u0001)-approximation for UFL under the super-set output setting. Finally in Section 5, we prove our\n\u0001)-lower bound on the approximation ratio for the same setting. All missing proofs will be\n\u2126(1/\ndeferred to the full version of the paper.\n3 Reducing General Metrics to Hierarchically Well-Separated Tree Metrics\nThe classic result of Fakcharoenphol, Rao and Talwar (FRT) [6] shows that any metric on n points\ncan be embedded into a distribution of metrics induced by hierarchically well-separated trees with\ndistortion O(log n). As in [8], this tree-embedding result is our starting point for our DP algorithm\nfor uniform UFL. To apply the technique, we \ufb01rst de\ufb01ne what is a hierarchically well-separated tree.\nDe\ufb01nition 3. For any real number \u03bb > 1, an integer L \u2265 1, a \u03bb-Hierarchically Well-Separated tree\n(\u03bb-HST) of depth L is an edge-weighted rooted tree T satisfying the following properties:\n\nEcostd(A( (cid:126)N ); (cid:126)N )\nminS\u2286V costd(S; (cid:126)N )\n\n(3a) Every root-to-leaf path in T has exactly L edges.\n\n(3b) If we de\ufb01ne the level of a vertex v in T to be L minus the number of edges in the unique\nroot-to-v path in T , then an edge between two vertices of level (cid:96) and (cid:96) + 1 has weight \u03bb(cid:96).\nGiven a \u03bb-HST T , we shall always use VT to denote its vertex set. For a vertex v \u2208 VT , we let (cid:96)T (v)\ndenote the level of v using the de\ufb01nition in (3b). Thus, the root r of T has level (cid:96)T (r) = L and every\nleaf v \u2208 T has level (cid:96)T (v) = 0. For every u, v \u2208 VT , de\ufb01ne dT (u, v) be the total weight of edges in\nthe unique path from u to v in T . So (VT , dT ) is a metric. With the de\ufb01nitions, we have:\nFact 4. Let u \u2208 VT be a non-leaf of T and v (cid:54)= u be a descendant of u, then\n\n\u03bb(cid:96)T (u)\u22121 \u2264 dT (u, v) \u2264 \u03bb(cid:96)T (u)\u22121\n\n\u03bb\u22121 \u2264 \u03bb(cid:96)T (u)\n\u03bb\u22121 .\n\nWe say a metric (V, d) is a \u03bb-HST metric for some \u03bb > 1 if there exists a \u03bb-HST T with leaves being\nV such that (V, d) \u2261 (V, dT|V ), where dT|V is the function dT restricted to pairs in V . Throughout\nthe paper, we guarantee that if a metric is a \u03bb-HST metric, the correspondent \u03bb-HST T is given. We\ngive the formal description of the FRT result as well how to apply it to reduce UFL on general metrics\nto that on O(1)-HST metrics in the full version of the paper.\nSpeci\ufb01cally, we shall prove the following theorem:\nTheorem 5. Let \u03bb > 1 be any absolute constant. If there exists an ef\ufb01cient \u0001-DP \u03b1tree(n, \u0001)-\napproximation algorithm A for UFL on \u03bb-HST\u2019s under the super-set output setting, then there exists\nan ef\ufb01cient \u0001-DP O(log n) \u00b7 \u03b1tree(n, \u0001)-approximation algorithm for UFL on general metrics under\nthe same setting.\n\n3\n\n\fIn Section 4, we shall show that it is possible to make \u03b1tree(n, \u0001) = O(1/\u0001):\nTheorem 6. For every small enough absolute constant \u03bb > 1, there is an ef\ufb01cient \u0001-DP O(1/\u0001)-\napproximation algorithm for UFL on \u03bb-HST metrics under the super-set output setting.\n\nCombining Theorems 5 and 6 will give our main theorem.\nTheorem 7 (Main Theorem). Given any UFL tuple (V, d, f, (cid:126)N ) where |V | = n and \u0001 > 0, there\nis an ef\ufb01cient \u0001-DP algorithm A in the super-set output setting achieving an approximation ratio of\nO( log n\n\n).\n\n\u0001\n\ndesign an \u0001-DP(cid:0)\u03b1tree = O(1/\u0001)(cid:1)-approximation algorithm for UFL instances on the metric (V, dT ).\n\n4\n\u0001-DP Algorithm with O(1/\u0001) Approximation Ratio for HST Metrics\nIn this section, we prove Theorem 6. Let \u03bb \u2208 (1, 2) be any absolute constant and let \u03b7 =\n\u03bb. We\nprove the theorem for this \ufb01xed \u03bb. So we are given a \u03bb-HST T with leaves being V . Our goal is to\nOur input vector is (cid:126)N = (Nv)v\u2208V , where Nv \u2208 Z\u22650 is the number of clients at the location v \u2208 V .\n\n\u221a\n\n4.1 Useful De\ufb01nitions and Tools\n\nBefore describing our algorithm, we introduce some useful de\ufb01nitions and tools. Recall that VT is the\nset of vertices in T and V \u2286 VT is the set of leaves. Since we are dealing with a \ufb01xed T in this section,\nwe shall use (cid:96)(v) for (cid:96)T (v). Given any u \u2208 VT , we use Tu to denote the sub-tree of T rooted at u.\nLet L \u2265 1 be the depth of T ; we assume L \u2265 log\u03bb(\u0001f ).2 We use L(cid:48) = max{0,(cid:100)log\u03bb(\u0001f )(cid:101)} \u2264 L to\ndenote the smallest non-negative integer (cid:96) such that \u03bb(cid:96) \u2265 \u0001f.\n\nWe extend the de\ufb01nition of Nu\u2019s to non-leaves u of T : For every u \u2208 VT \\V , let Nu =(cid:80)\n\nv\u2208Tu\u2229V Nv\nto be the total number of clients in the tree Tu.\nWe can assume that facilities can be built at any location v \u2208 VT (instead of only at leaves V ): On\none hand, this assumption enriches the set of valid solutions and thus only decreases the optimum\ncost. On the other hand, for any u \u2208 VT with an open facility, we can move the facility to any leaf v\nin Tu. Then for any leaf v(cid:48) \u2208 V , it is the case that d(v(cid:48), v) \u2264 2d(v(cid:48), u). Thus moving facilities from\nVT \\ V to V only incurs a factor of 2 in the connection cost.\nAn important function that will be used throughout this section is the following set of minimal\nvertices:\nDe\ufb01nition 8. For a set M \u2286 T of vertices in T , let\n\nmin-set(M ) := {u \u2208 M : \u2200v \u2208 Tu \\ {u}, v /\u2208 M}.\n\nFor every v, let we de\ufb01ne Bv := min{f, Nv\u03bb(cid:96)(v)}. This can be viewed as a lower bound on the cost\nincurred inside the tree Tv, as can be seen from the following claim:\nClaim 9. Let V (cid:48) \u2286 VT be a subset of vertices that does not contain an ancestor-descendant pair3.\n\nThen we have opt \u2265(cid:80)\n\nv\u2208V (cid:48) Bv.\n\n4.2 Base Algorithm for UFL without Privacy Guarantee\n\nBefore describing the \u0001-DP algorithm, we \ufb01rst give a base algorithm (Algorithm 1) without any privacy\nguarantee as the starting point of our algorithmic design. The algorithm gives an approximation ratio\nof O(1/\u0001); however, it is fairly simple to see that by making a small parameter change, we can achieve\nO(1)-approximation ratio. We choose to present the algorithm with O(1/\u0001)-ratio only to make it\ncloser to our \ufb01nal algorithm (Algorithm 2), which is simply the noise-version of the base algorithm.\nThe noise makes the algorithm \u0001-DP, while only incurring a small loss in the approximation ratio.\nRecall that we are considering the super-set output setting,where we return a set R of facilities, but\nonly open a set S \u2286 R of facilities using the following closest-facility rule: We connect every client\nto its nearest facility in R, then the set S \u2286 R of open facilities is the set of facilities in R with at\nleast 1 connected client.\n\nroot for T and let the old root be its child.\n\n2If this is not the case, we can repeat the following process many steps until the condition holds: create a new\n3This means for every two distinct vertices u, v \u2208 V (cid:48), u is not an ancestor of v\n\n4\n\n\fAlgorithm 1 UFL-tree-base(\u0001)\n1: L(cid:48) \u2190 max{0,(cid:100)log\u03bb(\u0001f )(cid:101)}\n\n2: Let M \u2190(cid:110)\n\nv \u2208 VT : (cid:96)(v) \u2265 L(cid:48) or Nv \u00b7 \u03bb(cid:96)(v) \u2265 f\n\nbe the set of marked vertices\n\n3: R \u2190 min-set(M )\n4: return R but only open S \u2286 R using the closest-facility rule.\n\nIn the base algorithm, M is the set of marked vertices in T and we call vertices not in M unmarked.\nAll vertices at levels [L(cid:48), L] are marked. Notice that there is a monotonicity property among vertices\nin VT : for two vertices u, v \u2208 VT with u being an ancestor of v, v is marked implies that u is marked.\nDue to this property, we call an unmarked vertex v \u2208 VT maximal-unmarked if its parent is marked.\nSimilarly, we call a marked vertex v \u2208 VT minimal-marked if all its children are unmarked (this is\nthe case if v is a leaf). So R is the set of minimal-marked vertices. Notice one difference between our\nalgorithm and that of [8]: we only return minimal-marked vertices, while [8] returns all marked ones.\nThis is one place where we can save a logarithmic factor, which requires more careful analysis.\nWe bound the facility and connection cost of the solution S given by Algorithm 1 respectively. Indeed,\nfor the facility cost, we prove some stronger statement. De\ufb01ne V \u25e6 = {u \u2208 Vt : Nu \u2265 1} be the set\nof vertices u with at least 1 client in Tu. We prove\nClaim 10. S \u2286 min-set(V \u25e6 \u2229 M ).\nThe stronger statement we prove about the facility cost of the solution S is the following:\nLemma 11. |min-set(V \u25e6 \u2229 M )| \u00b7 f \u2264 (1 + 1/\u0001)opt.\nNotice that Claim 10 and Lemma 11 imply that |S| \u00b7 f \u2264 O(1 + 1/\u0001)opt.\nNow we switch gear to consider the connection cost of the solution S and prove:\nLemma 12. The connection cost of S given by the base algorithm is at most O(1)opt.\n\n4.3 Guaranteeing \u0001-DP by Adding Noises\n\nIn this section, we describe the \ufb01nal algorithm (Algorithm 2) that achieves \u0001-DP without sacri\ufb01cing\nthe order of the approximation ratio. Recall that \u03b7 =\n\n\u03bb.\n\n\u221a\n\n(cid:111)\n\n(cid:111)\n\nAlgorithm 2 DP-UFL-tree(\u0001)\n1: L(cid:48) \u2190 max{0,(cid:100)log\u03bb(\u0001f )(cid:101)}\n2: for every v \u2208 VT with (cid:96)(v) < L(cid:48), de\ufb01ne \u02dcNv := Nv + Lap\n\n(cid:18)\n\nf\n\nc\u03b7L(cid:48)+(cid:96)(v)\n\n(cid:19)\n\n, where c = \u03b7\u22121\n\u03b72 .\n\nv \u2208 VT : (cid:96)(v) \u2265 L(cid:48) or \u02dcNv \u00b7 \u03bb(cid:96)(v) \u2265 f\n\nbe the set of marked vertices\n\n3: Let M \u2190(cid:110)\n\n4: R \u2190 min-set(M )\n5: return R but only open S \u2286 R using the closest-facility rule.\n\nWe give some intuitions on how we choose the noises in Step 1 of the Algorithm. Let us travel\nthrough the tree from level L(cid:48) down to level 0. Then the Laplacian parameter, which corresponds to\nthe magnitude of the Laplacian noise, goes up by factors of \u03b7. This scaling factor is carefully chosen\nto guarantee two properties. First the noise should go up exponentially so that the DP parameter\nonly depends on the noise on the highest level, i.e, level L(cid:48). Second, \u03b7 is smaller than the distance\nscaling factor \u03bb = \u03b72. Though the noises are getting bigger as we travel down the tree, their effects\nare getting smaller since they do not grow fast enough. Then essentially, the effect of the noises is\nonly on levels near L(cid:48).\nLemma 13. Algorithm 2 satis\ufb01es \u0001-DP property.\n\n4.4\n\nIncrease of cost due to the noises\n\nWe shall analyze how the noise affects the facility and connection costs. Let M 0, R0 and S0 (resp.\nM 1, R1 and S1) be the M, R and S generated by Algorithm 1 (resp. Algorithm 2). In the proof,\n\n5\n\n\fwe shall also consider running Algorithm 1 with input vector being 2 (cid:126)N instead of (cid:126)N. Let M(cid:48)0, R(cid:48)0\nand S(cid:48)0 be the M, R and S generated by Algorithm 1 when the input vector is 2 (cid:126)N. Notice that the\noptimum solution for input vector 2 (cid:126)N is at most 2opt. Thus, Lemma 11 implies |S(cid:48)0|\u00b7f = O(1/\u0001)opt.\nNotice that M 0, R0, S0, M(cid:48)0, R(cid:48)0 and S(cid:48)0 are deterministic while M 1, R1 and S1 are randomized.\nThe lemmas we shall prove are the following:\n\nLemma 14. E(cid:2)|S1| \u00b7 f(cid:3) \u2264 O(1/\u0001) \u00b7 opt.\n\nLemma 15. The expected connection cost of the solution S1 is O(1) times that of S0.\n\nThus, combining the two lemmas, we have that the expected cost of the solution S1 is at most\nO(1/\u0001)opt, \ufb01nishing the proof of Theorem 6. Indeed, we only lose an O(1)-factor for the connection\ncost as both factors in Lemma 11 and 15 are O(1). We then prove the two lemmas separately.\n\n4.4.1 Increase of facility costs due to the noise\n\nIn this section, we prove Lemma 14. A technical lemma we can prove is the following:\nClaim 16. Let M \u2286 VT and M(cid:48) = M \u222a {v} for some v \u2208 VT \\ M, then exactly one of following\nthree cases happens.\n\n(16a) min-set(M(cid:48)) = min-set(M ).\n(16b) min-set(M(cid:48)) = min-set(M ) (cid:93) {v}.\n(16c) min-set(M(cid:48)) = min-set(M ) \\ {u} \u222a {v}, where u \u2208 min-set(M ), v /\u2208 min-set(M ) and\n\nv is a descendant of u.\n\nProof of Lemma 14. Recall that V \u25e6 is the set of vertices u with Nu \u2265 1. We \ufb01rst focus on open\nfacilities in V \u25e6 in S1. Claim 16 implies that adding one new element to M will increase |min-set(M )|\nby at most 1. Thus, we have\n\n|min-set(M 1 \u2229 V \u25e6)| \u2212 |min-set(M(cid:48)0 \u2229 V \u25e6)| \u2264 |(M 1 \u2229 V \u25e6) \\ (M(cid:48)0 \u2229 V \u25e6)|\n\n=\n\nu \u2208 V \u25e6 : (cid:96)(u) < L(cid:48), 2Nu <\n\nf\n\n\u03bb(cid:96)(u)\n\n\u2264 \u02dcNu\n\n(cid:20)\n\nWe now bound the expectation of the above quantity. Let U\u2217 be the set of vertices u \u2208 V \u25e6 with\n(cid:96)(u) < L(cid:48) and Nu < f\n\n(cid:19)\n2\u03bb(cid:96)(u) . Then for every u \u2208 U\u2217, we have\n(cid:32)\n\u2265 f\nPr[u \u2208 M 1] = Pr\n\u03bb(cid:96)(u)\n\u2212 c\u03b7L(cid:48)\u2212(cid:96)(u)\nhave Bu = min(cid:8)f, Nu\u03bb(cid:96)(u)(cid:9) \u2265 \u03bb(cid:96)(u) for every u we are interested. Then,\n\n(2)\nWe bound f times the sum of (2), over all u \u2208 U\u2217. Notice that every u \u2208 V \u25e6 has Nu \u2265 1. So we\n\nf /(cid:0)c\u03b7L(cid:48)+(cid:96)(u)(cid:1)(cid:33)\n\n\u2212 f /(2\u03bb(cid:96)(u))\n\n(cid:21)\n(cid:33)\n\nc\u03b7L(cid:48)+(cid:96)(u)\n\nNv + Lap\n\n\u2264 1\n2\n\n(cid:32)\n\n(cid:18)\n\nexp\n\nexp\n\n2\n\n=\n\n1\n2\n\nf\n\n.\n\nf \u2264 1\n\u0001\n\n\u00b7 \u0001f \u00b7 Bu\n\u03bb(cid:96)(u)\nThe last inequality comes from \u0001f \u2264 \u03bbL(cid:48)\n. The equality used that \u03bb = \u03b72.\nWe group the u\u2019s according to (cid:96)(u). For each level (cid:96) \u2208 [0, L(cid:48) \u2212 1], we have\n\n\u00b7 \u03bbL(cid:48) \u00b7 Bu\n\u03bb(cid:96)(u)\n\n\u2264 1\n\u0001\n\nBu\n\u0001\n\n\u00b7 \u03b72(L(cid:48)\u2212(cid:96)(u)).\n\n=\n\n(cid:88)\n\n(cid:32)\n\u2212 c\u03b7L(cid:48)\u2212(cid:96)(u)\n\n(cid:33)\n\nf\n2\n\nexp\n\nu\u2208U\u2217:(cid:96)(u)=(cid:96)\n\n\u2264 1\n2\u0001\n(cid:96) exp(\u2212 cx(cid:96)\nwhere we de\ufb01ned x(cid:96) = \u03b7L(cid:48)\u2212(cid:96)(u) and c(cid:96) = x2\nholds since all u\u2019s in the summation are at the same level.\nTaking the sum over all (cid:96) from 0 to L(cid:48), we obtain\n\n\u03b72(L(cid:48)\u2212(cid:96)) exp\n\n2\n\n(3)\n\nBu \u2264 c(cid:96)\n2\u0001\n\nopt,\n\n(cid:32)\n\u2212 c\u03b7L(cid:48)\u2212(cid:96)\n\n2\n\n(cid:33) (cid:88)\n\nu as before\n\n2 ). The last inequality used Claim 9, which\n\n(cid:27)(cid:12)(cid:12)(cid:12)(cid:12) .\n\n(cid:12)(cid:12)(cid:12)(cid:12)(cid:26)\n\n(cid:88)\n\nf\n\nu\u2208U\u2217\n\nPr[u \u2208 M 1] \u2264 opt\n2\u0001\n\nc(cid:96) =\n\nopt\n2\u0001\n\n\u00b7 L(cid:48)\u22121(cid:88)\n\n(cid:96)=0\n\n(cid:96) exp(\u2212 cx(cid:96)\nx2\n2\n\n).\n\n\u00b7 L(cid:48)\u22121(cid:88)\n\n(cid:96)=0\n\n6\n\n\fNotice that {x(cid:96) : (cid:96) \u2208 [0, L(cid:48) \u2212 1]} is exactly {\u03b7, \u03b72,\u00b7\u00b7\u00b7 , \u03b7L(cid:48)}. It is easy to see summation is bounded\nby a constant for any constant c. Thus, the above quantity is at most O(1/\u0001)opt. Therefore, we\nproved\n\nf \u00b7 E(cid:2)|min-set(M 1 \u2229 V \u25e6)| \u2212 |min-set(M(cid:48)0 \u2229 V \u25e6)|(cid:3) \u2264 O(1/\u0001) \u00b7 opt.\n\nNotice that Lemma 11 says that f \u00b7 |min-set(M(cid:48)0 \u2229 V \u25e6)| \u2264 O(1/\u0001)opt. Thus f \u00b7 E[|min-set(M 1 \u2229\nV \u25e6)|] \u2264 O(1/\u0001)opt.\nThen we take vertices outside V \u25e6 into consideration. Let U = min-set(M 1 \u2229 V \u25e6). Then S1 \u2286\nR1 = min-set(U \u222a (VT \\ V \u25e6)). To bound the facility cost of S1, we start with the set U(cid:48) = U\nand add vertices in VT \\ V \u25e6 (these are vertices u with Nu = 0) to U(cid:48) one by one and see how\nthis changes min-set(U(cid:48)). By Claim 16, adding a vertex Nv = 0 to U(cid:48) will either not change\nmin-set(U(cid:48)), or add v to min-set(U(cid:48)), or replace an ancestor of v with v. In all the cases, the set\nmin-set(U(cid:48)) \u2229 V \u25e6 can only shrink. Thus, we have R1 \u2229 V \u25e6 \u2286 min-set(U ) = min-set(M 1 \u2229 V \u25e6).\nWe have E[|R1 \u2229 V | \u00b7 f ] \u2264 O(1/\u0001) \u00b7 opt.\nThus, it suf\ufb01ces to bound the expectation of |S \\ V \u25e6|\u00b7 f. Focus on some u \u2208 VT with Nu = 0. Notice\nthat u /\u2208 S if (cid:96)(u) \u2265 L(cid:48). So, we assume (cid:96)(u) < L(cid:48). In this case there is some leaf v \u2208 V with Nv > 0\nsuch that u is the closest point in R to v. So v is not a descendant of u. Let u(cid:48) be the ancestor of v that\nis at the same level at u and de\ufb01ne \u03c0(u) = u(cid:48). Then (cid:96)(\u03c0(u)) = (cid:96)(u). Moreover, u is also the closest\npoint in R to u(cid:48), implying that \u03c0 is an injection. For every u, we can bound f as in (3), but with Bu\nu:Nu=0,(cid:96)(u)=(cid:96) B\u03c0(u) \u2264 opt\nfor every (cid:96) \u2208 [0, L(cid:48) \u2212 1] by Claim 9.\n\nreplaced by B\u03c0(u). Then the above analysis still works since we have(cid:80)\n\n4.4.2 Increase of connection cost due to the noise\n\nNow we switch gear to consider the change of connection cost due to the noise.\nProof of Lemma 15. Focus on a vertex v at level (cid:96) and suppose v \u2208 S0 and some clients are connected\nto v in the solution produced by Algorithm 2. So, we have Nv \u2265 f\n\u03bb(cid:96) . Let the ancestor of v (including\nv itself) be v0 = v, v1, v2,\u00b7\u00b7\u00b7 from the bottom to the top. Then the probability that v0 /\u2208 M 0 is\nat most 1/2 and in that case the connection cost increases by a factor of \u03bb. The probability that\nv0, v1 /\u2208 M 0 is at most 1/4, and in that case the cost increases by a factor of \u03bb2 and so on. As a\nresult, the expected scaling factor for the connection cost due to the noise is at most\n\n(cid:18) \u03bb\n\n(cid:19)i\n\n\u221e(cid:88)\n\n1\n\n2i \u00b7 \u03bbi =\n\n2\n\ni=1\n\n= O(1).\n\nThus, the connection cost of the solution S1 is at most a constant times that of S0. This is the place\nwhere we require \u03bb < 2.\n\n5 Lower Bound of UFL for HST Metric\n\n\u03bb\u22121 for some integer L.\n\nIn this section, we prove an \u2126(1/\n\u0001) lower bound on the approximation ratio of any algorithm for\nUFL in the super-set setting under the \u0001-DP model. The metric we are using is the uniform star-metric:\nthe shortest-path metric of a star where all edges have the same length. We call the number of edges\nin the star its size and the length of these edges its radius. By splitting edges, we can easily see that\nthe metric is a \u03bb-HST metric for a \u03bb > 1, if the radius is \u03bbL\nThe main theorem we are going to prove is the following:\nTheorem 17. There is a constant c > 0 such that the following holds. For any small enough\n\u0001 < 1, f > 0 and suf\ufb01ciently large integer n that depends on \u0001, there exists a set of UFL instances\n{(V, d, f, (cid:126)N )} (cid:126)N , where (V, d) is the uniform-star metric of size n and radius\n\u0001f, and every instance\nin the set has n \u2264 | (cid:126)N|1 \u2264 n/\u0001, such that the following holds: no \u0001-DP algorithm under the super-set\nsetting can achieve c 1\u221a\nProof. Throughout the proof, we let m = 1/\u0001 and we assume m is an integer. We prove Theorem 17\nin two steps, \ufb01rst we show the lower bound on an instance with a 2-point metric, but non-uniform\nfacility costs. Then we make the facility costs uniform by combining multiple copies of the 2-point\nmetric into a star metric.\n\n\u0001-approximation for all the instances in the set.\n\n\u221a\n\n\u221e(cid:88)\n\ni=0\n\n\u221a\n\n7\n\n\fConsider the instance shown in Figure 1a where V = {a, b} and d(a, b) =\n\u0001f. The facility costs\nfor a and b are respectively f and 0. Thus, we can assume the facility b is always open. All the clients\nare at a, and the number N of clients is promised to be an integer between 1 and m. We show that\n\u221a\nfor this instance, no \u0001-DP algorithm in the super-set output setting can distinguish between the case\nwhere N = 1 and that N = m with constant probability; this will establish the \u2126(\nm) lower bound.\n\n\u221a\n\n(a)\n\n(b)\n\n(c)\n\nFigure 1: Instance for the lower bound.\n\nObviously, there are only 2 solutions for any instance in the setting: either we open a, or we do not.\nSince we are promised there is at least 1 client, the central curator has to reveal whether we open a or\nnot, even in the super-set output setting: If we do not open a, then we should not include a in the\nreturned set R since otherwise the client will think it is connected to a; if we open a, then we need to\ninclude it in the returned set R since all the clients need to be connected to a.\nLet Di be the scenario where we have N = i clients at a, where i \u2208 [m]. Then the cost of the two\nsolutions for the two scenarios D1 and Dm are listed in the following table:\n\nnot open a\n\n\u221a\n\u221a\nf /\n\nm\nmf\n\nopen a\n\nf\nf\n\nD1\nDm\n\n\u221a\n\u221a\nThus, if the data set is D1, we should not open a; if we opened, we\u2019ll lose a factor of\nm. If the data\nset is Dm, then we should open a; if we did not open, then we also lose a factor of\nm.\n\u221a\nNow consider any \u0001-DP algorithm A. Assume towards the contradiction that A achieves 0.2\nm\napproximation ratio. Then, under the data set D1, A should choose not to open a with probability at\nleast 0.8. By the \u0001-DP property, under the data set Dm, A shall choose not to open a with probability\nat least 0.8e\u2212(m\u22121)\u0001 > 0.8/e \u2265 0.2. Then under the data set Dm, the approximation ratio of A is\n\u221a\nmore than 0.2\nIndeed, later we need an average version of the lower bound as follows:\n\nm, leading to a contradiction.\n\n\u221a\nm\u221a\nm + 1\n\nE cost(A(1); 1) +\n\n(4)\nwhere c is an absolute constant, A(N ) is the solution output by the algorithm A when there are\nN clients at a, and cost(A(N ); N ) is the cost of the solution under the input N. Our argument\n\u221a\nabove showed that either E cost(A(1); 1) \u2265 \u2126(0.2\nm = 0.2f, or E cost(A(N ); N ) \u2265\n\u221a\n0.2\n\n\u221a\nmf. In either case, the left side of (4) is at least 0.2\n\n\u221a\nm \u00b7 f = 0.2\n\n\u221a\nm) \u00b7 f /\n\n\u221a\nm+1 \u2265 cf if c is small.\n\nmf\n\nE cost(A(m); m) \u2265 cf,\n\n1\u221a\nm + 1\n\nThe above proof almost proved Theorem 17 except that we need to place a free open facility at\nlocation b. To make the facility costs uniform, we can make multiple copies of the locations a, while\nonly keeping one location b; this is exactly a star metric (see Figure 1b). The costs for all facilities\nare f. However, since there are so many copies of a, the cost of f for opening a facility at b is so\nsmall and thus can be ignored. Then, the instance essentially becomes many separate copies of the\n2-point instances we described (see Figure 1c).\nHowever, proving that the \u201cparallel repetition\u201d instance in Figure 1c has the same lower bound\nas the original two-point instance is not so straightforward. Intuitively, we can imagine that the\ncentral curator should treat all copies independently: the input data for one copy should not affect\nthe decisions we make for the other copies. However, it is tricky to prove this. Instead, we prove\nTheorem 17 directly by de\ufb01ning a distribution over all possible instances and argue that an \u0001-DP\nalgorithm must be bad on average.\n\n8\n\nbaf/\u221amcost=0cost=fa1a2\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7an\u22121anbcost=ff/\u221amcost=fcost=fcost=fcost=f\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7b1b2b3bna1a2a3an\u221e\u221ecost=0f/\u221amcost=0cost=0cost=0cost=fcost=fcost=fcost=f\fDue to the page limit, the detailed analysis is left to the full vesion of the paper.\n\nAcknowledgements\n\nYunus Esencayi is supported in part by NSF grants CCF-1566356. Part of the work was done when\nDi Wang and Marco Gaboardi were visiting the Simons Institute of the Theory for Computing. Marco\nGaboardi is supported in part by NSF through grant CCF-1718220. Di Wang is supported in part\nby NSF through grant CCF-1716400. Shi Li is supported in part by NSF grants CCF-1566356,\nCCF-1717138 and CCF-1844890.\n\nReferences\n[1] Vijay Arya, Naveen Garg, Rohit Khandekar, Adam Meyerson, Kamesh Munagala, and Vinayaka\nPandit. Local search heuristics for k-median and facility location problems. SIAM Journal on\ncomputing, 33(3):544\u2013562, 2004.\n\n[2] Maria-Florina Balcan, Travis Dick, Yingyu Liang, Wenlong Mou, and Hongyang Zhang.\nDifferentially private clustering in high-dimensional euclidean spaces. In Proceedings of the\n34th International Conference on Machine Learning-Volume 70, pages 322\u2013331. JMLR. org,\n2017.\n\n[3] Adrian Rivera Cardoso and Rachel Cummings. Differentially private online submodular\nminimization. In The 22nd International Conference on Arti\ufb01cial Intelligence and Statistics,\npages 1650\u20131658, 2019.\n\n[4] Moses Charikar and Sudipto Guha. Improved combinatorial algorithms for the facility location\nand k-median problems. In 40th Annual Symposium on Foundations of Computer Science,\npages 378\u2013388. IEEE, 1999.\n\n[5] Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to\n\nsensitivity in private data analysis. In TCC, pages 265\u2013284. Springer, 2006.\n\n[6] Jittat Fakcharoenphol, Satish Rao, and Kunal Talwar. A tight bound on approximating arbitrary\n\nmetrics by tree metrics. Journal of Computer and System Sciences, 69(3):485\u2013497, 2004.\n\n[7] Dan Feldman, Amos Fiat, Haim Kaplan, and Kobbi Nissim. Private coresets. In Proceedings of\nthe forty-\ufb01rst annual ACM symposium on Theory of computing, pages 361\u2013370. ACM, 2009.\n\n[8] Anupam Gupta, Katrina Ligett, Frank McSherry, Aaron Roth, and Kunal Talwar. Differentially\nIn Proceedings of the twenty-\ufb01rst annual ACM-SIAM\nprivate combinatorial optimization.\nsymposium on Discrete Algorithms, pages 1106\u20131125. Society for Industrial and Applied\nMathematics, 2010.\n\n[9] Justin Hsu, Zhiyi Huang, Aaron Roth, Tim Roughgarden, and Zhiwei Steven Wu. Private\n\nmatchings and allocations. SIAM Journal on Computing, 45(6):1953\u20131984, 2016.\n\n[10] Justin Hsu, Zhiyi Huang, Aaron Roth, and Zhiwei Steven Wu. Jointly private convex pro-\ngramming. In Proceedings of the twenty-seventh annual ACM-SIAM symposium on Discrete\nalgorithms, pages 580\u2013599. Society for Industrial and Applied Mathematics, 2016.\n\n[11] Zhiyi Huang and Xue Zhu. Near optimal jointly private packing algorithms via dual multiplica-\ntive weight update. In Proceedings of the Twenty-Ninth Annual ACM-SIAM Symposium on\nDiscrete Algorithms, pages 343\u2013357. Society for Industrial and Applied Mathematics, 2018.\n\n[12] Zhiyi Huang and Xue Zhu. Scalable and jointly differentially private packing. arXiv preprint\n\narXiv:1905.00767, 2019.\n\n[13] Kamal Jain and Vijay V Vazirani. Approximation algorithms for metric facility location and\nk-median problems using the primal-dual schema and lagrangian relaxation. Journal of the\nACM (JACM), 48(2):274\u2013296, 2001.\n\n9\n\n\f[14] Michael Kearns, Mallesh Pai, Aaron Roth, and Jonathan Ullman. Mechanism design in large\ngames: Incentives and privacy. In Proceedings of the 5th conference on Innovations in theoretical\ncomputer science, pages 403\u2013410. ACM, 2014.\n\n[15] Shi Li. A 1.488 approximation algorithm for the uncapacitated facility location problem. In\nInternational Colloquium on Automata, Languages, and Programming, pages 77\u201388. Springer,\n2011.\n\n[16] Shi Li. On facility location with general lower bounds. In Proceedings of the Thirtieth Annual\nACM-SIAM Symposium on Discrete Algorithms, pages 2279\u20132290. Society for Industrial and\nApplied Mathematics, 2019.\n\n[17] Marko Mitrovic, Mark Bun, Andreas Krause, and Amin Karbasi. Differentially private submod-\nular maximization: data summarization in disguise. In Proceedings of the 34th International\nConference on Machine Learning-Volume 70, pages 2478\u20132487. JMLR. org, 2017.\n\n[18] Kobbi Nissim, Rann Smorodinsky, and Moshe Tennenholtz. Approximately optimal mechanism\ndesign via differential privacy. In Innovations in Theoretical Computer Science 2012, Cambridge,\nMA, USA, January 8-10, 2012, pages 203\u2013213, 2012.\n\n[19] David B Shmoys and KI Aardal. Approximation algorithms for facility location problems,\n\nvolume 1997. Utrecht University: Information and Computing Sciences, 1997.\n\n[20] Jean-Michel Thizy. A facility location problem with aggregate capacity. INFOR: Information\n\nSystems and Operational Research, 32(1):1\u201318, 1994.\n\n10\n\n\f", "award": [], "sourceid": 4587, "authors": [{"given_name": "Yunus", "family_name": "Esencayi", "institution": "State University of New York at Buffalo"}, {"given_name": "Marco", "family_name": "Gaboardi", "institution": "Univeristy at Buffalo"}, {"given_name": "Shi", "family_name": "Li", "institution": "University at Buffalo"}, {"given_name": "Di", "family_name": "Wang", "institution": "State University of New York at Buffalo"}]}