{"title": "On Differentially Private Graph Sparsification and Applications", "book": "Advances in Neural Information Processing Systems", "page_first": 13399, "page_last": 13410, "abstract": "In this paper, we study private sparsification of graphs. In particular, we give an algorithm that given an input graph, returns a sparse graph which approximates the spectrum of the input graph while ensuring differential privacy. This allows one to solve many graph problems privately yet efficiently and accurately. This is exemplified with application of the proposed meta-algorithm to graph algorithms for privately answering cut-queries, as well as practical algorithms for computing {\\scshape MAX-CUT} and {\\scshape SPARSEST-CUT} with better accuracy than previously known. We also give the first efficient private algorithm to learn Laplacian eigenmap on a graph.", "full_text": "On Differentially Private Graph Sparsi\ufb01cation and\n\nApplications\n\nRaman Arora\n\nJohns Hopkins University\n\narora@cs.jhu.edu\n\nJalaj Upadhyay\nRutgers University\n\njalaj.kumar.upadhyay@gmail.com\n\nAbstract\n\nIn this paper, we study private sparsi\ufb01cation of graphs. In particular, we give an\nalgorithm that given an input graph, returns a sparse graph which approximates\nthe spectrum of the input graph while ensuring differential privacy. This allows\none to solve many graph problems privately yet ef\ufb01ciently and accurately. This is\nexempli\ufb01ed with application of the proposed meta-algorithm to graph algorithms\nfor privately answering cut-queries, as well as practical algorithms for computing\nMAX-CUT and SPARSEST-CUT with better accuracy than previously known.\nWe also give an ef\ufb01cient private algorithm to learn Laplacian eigenmap on a graph.\n\n1\n\nIntroduction\n\nData from social and communication networks have become a rich source to gain useful insights\ninto the social, behavioral, and information sciences. Such data is naturally modeled as observations\non a graph, and encodes rich, \ufb01ne-grained, and structured information. At the same time, due to\nthe seamless nature of data acquisition, often collected through personal devices, the individual\ninformation content in network data is often highly sensitive. This raises valid privacy concerns\npertaining the analysis and release of such data. We address these concerns in this paper by presenting\na novel algorithm that can be used to publish a succinct differentially private representation of network\ndata with minimal degradation in accuracy for various graph related tasks.\nThere are several notions of differential privacy one can consider in the setting described above.\nDepending on privacy requirements, one can consider edge level privacy that renders two graphs\nthat differ in a single edge as in-distinguishable based on the algorithm\u2019s output; this is the setting\nstudied in many recent works [9, 19, 25, 54]. Alternatively, one can require node-level privacy which\npreserves privacy of each node, which has been the focus in [10, 11, 28, 44]. In this paper, we\nconsider settings where nodes are known public entities and the edges represent sensitive or private\nevents and attributes relating two nodes.\nIn particular, we consider the following notion of differential privacy. We say that an algorithm that\ntakes a graph as input and returns a graph, is differentially private if given any two graphs that differ\nin a single edge by weight at most one 1, the output of the algorithm does not change by much (see\nSection 3 for a formal de\ufb01nition).\nGiven the ubiquitous nature of the problem, several recent works have studied various graph problems\nwithin the framework of differential privacy. These include the works on differentially private\nalgorithms for computing degree distribution [28, 27, 44], on subgraph counting [40, 45], on private\nMIN-CUT [22] and on estimating parameters of generative graphical models [39]. Each of the works\nreferenced above present algorithms that are tailor-made for a speci\ufb01c graph problems. We, on the\n1This is the standard neighboring relation used in Blocki et al. [9] (and references therein). The underlying\nreason for considering this privacy notion is as follows: since we do not restrict the weight on the graph, presence\nor absence of an edge, as required in standard edge-level privacy, can change the output drastically.\n\n33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada.\n\n\fother hand, are interested in understanding the problem more generally. We pose the following\nquestion: given an input graph, can we ef\ufb01ciently generate a succinct yet private representation that\nallows us to simultaneously solve multiple graph-related tasks accurately?\nTwo popular representations of graphs that succinctly capture several graph properties include spectral\nsparsi\ufb01cation and cut sparsi\ufb01cation. Spectral sparsi\ufb01cation [52] is the following problem: given a\n\n8x 2 Rn,\n\nx \uf8ff (1+\")x>LGx,\n\n(1\")x>LGx \uf8ff x>LeG\n\ngraph G with n nodes, output a subgraph eG of G such that (1 \")LG LeG (1 + \")LG, i.e.,\n(1)\nwhere LG is the Laplacian of the graph G (see De\ufb01nition 1). This is a generalization of cut sparsi\ufb01ca-\ntion [8], where x is restricted to binary vector. Spectral sparsi\ufb01cation of a graph is a fundamental\nproblem that has found application in randomized linear algebra [13, 33], graph problems [29], linear\nprogramming [34], and mathematics [37, 51].\nThere are many non-private algorithms for computing spectral sparsi\ufb01cation of graphs [2, 7, 35,\n36, 50, 52]. However, to the best of our knowledge, there is no prior work on differentially private\ngraph sparsi\ufb01cation. This paper initiates this study by formally stating the goal of private graph\nsparsi\ufb01cation and presents the \ufb01rst algorithm for ef\ufb01ciently computing a graph sparsi\ufb01cation. The\nmain contributions of this paper are as follows.\n\n1. We show that differential privacy is not achievable under the traditional notion of spectral\nsparsi\ufb01cation of graphs since the output itself may reveal information about the edges.\nFurthermore, we put forth an alternate but a well-posed formulation of differentially private\ngraph sparsi\ufb01cation problem.\n\n2. We give an ef\ufb01cient algorithm that outputs a private sparse graph with O(n/\"2) edges. Since\nour output is a graph and preserves the spectrum of the input graph, we can solve many\ngraph related combinatorial problems ef\ufb01ciently while preserving differential privacy.\n\nThe works most closely related to that of ours are that of Blocki et al. [9], Dwork et al. [19], and Gupta\net al. [23]. Blocki et al. [9] and Dwork et al. [19] give an algorithm that returns a symmetric matrix\nthat may not correspond to a graph but can be used to answer cut queries accurately while Gupta\net al. [23] output a private graph that approximates cut functions, but cannot approximate spectral\nproperties. Our algorithm for computing private sparse graph not only solves the two problems\nsimultaneously but also improves upon both of these works.\nSpectral sparsi\ufb01cation of graphs \ufb01nds many applications including, but not limited to, spectral\nclustering, heat kernel computations, separators, etc. Since differential privacy is preserved under\npost-processing, our result can be used in these applications to get a private algorithm for these tasks\n(see Table 1 in Section 4 for more details). On top of these improvements, we can leverage the fact\nthat our output is a graph that approximates the spectrum of the input graph to ef\ufb01ciently compute\nLaplacian eigenmap, a useful technique used in manifold learning (see Section 4 for more details).\n\n2 Preliminaries and Notations\n\n(EG)e,v :=8<:\n\nThe central object of interest in this paper is the Laplacian of a graph.\nDe\ufb01nition 1 (Laplacian of graph). Let G be an undirected graph with n vertices, m edges, and edge\nweights we. Consider an arbitrary orientation of edges of G, and let EG be the signed edge adjacency\nmatrix of G given by\n\n+pwe\npwe\n0\nThe Laplacian of G is de\ufb01ned as LG = E>\nG EG. Equivalently, LG = DG AG, where AG is the\nadjacency matrix and DG is the degree matrix. One can verify that LG1 = 1>LG = 0, where 1\ndenotes all 1 vector and 0 denotes all 0 vector.\nFor a vertex u 2 V , let u 2{ 0, 1}n denote the row vector with 1 only at the u-th coordinate. Lets\nrepresent the edges of the graph with vectors b1, . . . , bm 2 {1, 0, 1}n such that be = u v\nwhere the edge e connects u, v 2 V . Then, LG =Pe2EG\nb>e be. The spectral sparsi\ufb01cation of a\n\nif v is e\u2019s head\nif v is e\u2019s tail\notherwise\n\n.\n\n2\n\n\fgraph can be casted as the following linear algebraic problem: given a sparsity parameter s and row\nvectors b1, . . . , bm, \ufb01nd a set of scalars \u23271, . . . ,\u2327 m such that |{\u2327i : \u2327i 6= 0}|\uf8ff s and (1 \")LG \nPe2EG\n\u2327eb>e be (1+\")LG. For any graph G with edge-adjacency matrix EG, the effective resistance\n(also known as leverage score) of the edge ei 2 EG is de\ufb01ned ase\u2327i := e>i (E>\nGei.\nIt is well known that by sampling the edges (rows of EG) of G according to its leverage score, we\nobtain a graph eG such that (1 \")LG LeG (1 + \")LG (for example, see [50]).\nNotations. We use the notation (A | B) to denote the matrix formed by appending the columns\nof matrix B to that of matrix A. We denote by A\u2020 the Moore-Penrose pseudoinverse of A and by\nkAk2 its spectral norm. We use caligraphic letters to denote graphs, VG to denote the vertices of G\nand EG to denote the edges of G. We drop the subscript when it is clear from the context. We use the\nsymbol Kn to denote an n vertex complete graph and Ln to denote its Laplacian. For any S, T \u2713 VG,\nthe size of cut between S and T , denoted by G(S, T ), is the sum of weight of the edges that are\npresent between S and T . When T = V \\S, we denote the size of cut between S and V \\S by G(S).\nFor a set S \u2713 [n], we use the notation 1S =Pi2S ei, where {e1,\u00b7\u00b7\u00b7 , en} denote the standard basis.\n\nG EG)\u2020ei = e>i L\u2020\n\n3 Differentially Private Graph Sparsi\ufb01cation\n\nWe begin by noting that differential privacy is incompatible with the requirements of spectral\nsparsi\ufb01cation in equation (1), because if we output a sparse subgraph of the input graph, then it will\n\nleak information about eO(n) edges present in the graph. This motivates us to consider the following\n\u201crelaxation\" of the spectral sparsi\ufb01cation problem.\nDe\ufb01nition 2 ((\", \u21b5, , n)-Private Spectral Sparsi\ufb01cation). Let G be the set of all n vertex positive\nweighted graphs. We are interested in designing an ef\ufb01cient algorithm M : G ! G such that\n\n1. (Privacy) For all graphs G,G0 2 G that differ in only one edge by weight 1, and all possible\nmeasurable S \u2713 G, Pr[M(G) 2 S] \uf8ff e\u21b5Pr[M(G0) 2 S] + .\n2. (Sparsity) M(G) has at most eO(n) edges, and\n3. (Spectral approximation) eG M (G) satis\ufb01es (\uf8ffLG \u21e3L n) LeG \u2318LG + \u21e0Ln where\n\nfunctions \u2318, \u21e0, \uf8ff and \u21e3 dependent on input parameters n, \", \u21b5, .\n\nThe function \u21e3 and \u21e0 can be seen as the distortion we are willing to accept to preserve privacy. That\nis, we would like \u21e3 and \u21e0 to be as small as possible. Informally, we view adding Ln as introducing\nplausible deniability pertaining to the presence of an edge in the output; it could be coming from\neither G or the complete graph.\nThe choice of Ln is arbitrary and for simplicity. Our choice of Ln is motivated by the fact that an\nn vertex unweighted complete graph is the same irrespective of the input graph once the number\nof vertices is \ufb01xed. We can instead state item 3 by replacing Ln by Laplacian of any graph whose\nedges are independent of any function of G. For example, we can use a d-regular expander instead\nof Kn; however, this would not change our result as one can prove that a d-regular expander are\nspectral sparsi\ufb01cation of Kn. In the de\ufb01nition, we consider edge level privacy, a setting studied in\nmany recent works [9, 19, 25, 54]2 .\nWe \ufb01rst enumerate why previous work do not suf\ufb01ce. The two works most related work to ours is by\nBlocki et al. [9] and Upadhyay [54] used random projection on the edge-adjacency matrix of graph G\nto output a matrix, R. One can show that the spectrum of R>R approximates the spectrum of LG\nif the dimension of random projection is high enough. It is claimed in Blocki et al. [9] that their\noutput is a sanitized graph. However, even though their output is a Laplacian, one major drawback\nof random projection is that it does not preserve the structure of the matrix, which in our case is a\nedge-incidence matrix of a graph (we refer the readers to [13, 55] for more discussion on bene\ufb01ts and\npitfalls of random projections). Likewise, the output of Dwork et al. [19] is a full rank matrix (and\n2Note that the number of non-zero singular values of n vertex graph can be at most n 1 [20], where n is\nthe number of nodes in the graphs. That is, if we consider node-level privacy, then the Laplacian of a graph G\nhas at most n singular values and that for neighboring graph G0 is at most n 1 singular values. Thus outputting\nany spectral sparsi\ufb01cation would not preserve privacy.\n\n3\n\n\fhence not a Laplacian) that is not positive semi-de\ufb01nite matrix. Consequently, their result cannot be\nused for spectral sparsi\ufb01cation and other applications considered in this paper. On the other hand,\nGupta et al. [23] only approximates cut functions and not the spectrum.\nThe existing techniques for graph sparsi\ufb01cation are either deterministic [7]3 or use importance\nsampling that depends on the graph itself [50, 52]. A popular approach (and the one we use) involves\nsampling each edge with probability proportional to its effective resistance. We show that effective\nresistance based sampling can be done privately, albeit not trivially.\nIn order to comprehend the issues with private sampling based on the effective resistance, consider\ntwo neighboring graphs, G and G0, such that G has two connected components and G0 has an edge\ne between the two connected components of G. No sparsi\ufb01er for G will have the edge e; however,\nevery sparsi\ufb01er for G0 has to contain the edge e. This allows one to easily differentiate the two cases.\nFurthermore, we show that the effective resistance is not Lipschitz smooth, so we cannot hope to\nprivately compute effective resistance through output perturbation. One could argue that we can\ninstead use (a variant of) smooth sensitivity framework [40], but it is not clear which function is a\nsmooth bound on the sensitivity of effective resistance.\n\n3.1 A High-level Overview of Our Algorithm\n\nAs we noted earlier, spectral sparsi\ufb01cation of a graph can be posed as a linear algebra problem and\ncomputing effective resistance suf\ufb01ces for private sparsi\ufb01cation of graphs. We propose an algorithm\nto sample using privately computed effective resistance of edges. Our algorithm is based on the\nfollowing key ideas:\n\n1. Private sketch of the left and right singular vectors of the edge-adjacency matrix is enough\nto compute all the effective resistances privately and a (possibly dense) private graph, Gint,\nthat approximates the spectrum of the input graph.\n2. We then sparsify Gint to output a graph with O(n/\"2) edges with a small depreciation in the\n\nspectral approximation using any known non-private spectral sparsi\ufb01er.\n\n. For this, we sketch\n\nto sketch the right and left singular space so as to preserve differential privacy. Combined together\n\nComputing effective resistance privately. We use the input perturbation technique (and its variants)\n\ufb01rst introduced by Blocki et al. [9] to compute effective resistances privately. The scalars {\u23271, . . . ,\u2327 m}\nand Gint are then computed using these effective resistances. Blocki et al. [9] \ufb01rst overlay a weighted\ncomplete graph Kn on the input graph G to get a graph bG and then multiply its edge-adjacency\nwith a Gaussian matrix, say N. We view this algorithm as a private sketching algorithm,\nmatrix EbG\nwhere the sketch is R := N EbG\n. We prove in the supplementary material that if N is of appropriate\ndimension as chosen in Step 2 of Algorithm 1, then (1 \")LG E>\nN>N EbG LG. However, R\nbG\ndoes not contains enough information to estimate the effective resistances of EbG\nthe left singular space as L :=EbG\n| wI M for a Gaussian matrix M. We use different methods\nL and R has enough statistics to privately estimate effective resistance,e\u2327e, of each edge e using a\nsimple linear algebra computation (Step 2 of Algorithm 1).\nConstructing Gint. We overlay a weighted Kn in order to privately sketch the left singular space\nleading ton\n2) using the computed\neffective resistance as in Step 2 of Algorithm 1. Let eG0 be the graph formed by overlaying a weighted\ncomplete graph with edge weights sampled i.i.d. from a Gaussian distribution on bG. We cannot use\nexisting non-private algorithms for spectral sparsi\ufb01cation on graph eG0 as it may have negative weight\nHere C is the convex cone of the Laplacian of graphs with positive weights. Note that, SDP-1 has\na solution with = O( w\nn ). This is because the original graph G achieves this value. Since, the\nsemide\ufb01nite program requires to output a graph \u00afG with Laplacian L \u00afG, we are guaranteeed that its\n3Deterministic algorithms cannot be differentially private. On the other hand, if not done carefully, the\n\n2 effective resistances. We de\ufb01ne a set of scalarse\u23271, . . . ,e\u2327(n\n\nedges. To get a sanitized graph on which we can perform sparsi\ufb01cation, we solve the following:\n\nSDP-1 : min : L \u00afG LeG0 Ln, LeG0 L \u00afG Ln, L \u00afG 2C .\n\nsampling probability that depends on the graph can itself leak privacy.\n\n4\n\n\f2)\u21e5(n\n\nn EKn.\n\n2: Set w = O( 1\n\nn ), Nij \u21e0N (0, \"2\n, where EbG\n\nAlgorithm 1 PRIVATE-SPARSIFY(G,\", (\u21b5, ))\nInput: An n vertex graph G = (VG, EG), privacy parameters: (\u21b5, ), sparsi\ufb01cation parameter: \".\nOutput: A graph eG.\n1: Initialization. Set m =n\n2, p = m + n. Sample random Gaussian matrices M 2 Rp\u21e5n, N 2\nn ), and Qij \u21e0N (0, \"2\nRn/\"2\u21e5m, Q 2 Rp\u21e5p/\"2, such that Mij \u21e0N (0, 1\np ).\n\u21b5\"pn log n log(1/) log (1/)).\n| wI M, R := N EbG\n3: Compute L :=EbG\nnEG +p w\n4: De\ufb01nee\u2327i = LiR>R\u2020 L>i for every i 2 [m] and pi = mince\u2327i\"2 log(n/), 1 .\n5: Construct a diagonal matrix D 2 R(n\n6: Construct a complete graph H with edge weights i.i.d. sampled from N (0, 12 log(1/)\n7: Construct eG0 such that LeG0 = LbG\n+ LH. Solve the SDP-1 to get a graph \u00afG.\n8: Output eG formed by running the algorithm of Lee and Sun [35] on E>\u00afG\nDE \u00afG.\noutput, \u00afG, will have = O( w\nPee\u2327eu>e ue, where ue are the edges of \u00afG. The graph Gint can be a dense graph, but we show that it\napproximates the spectrum of the input graph up to eO( w\n\nn ). The Laplacian of the graph Gint is then constructed by computing\nn ) additive error. To reduce the number of\nedges, we then run any existing non-private spectral sparsi\ufb01cation algorithm to get the \ufb01nal output.\n\n2) whose non-zero diagonal entries are Dii := p1\n\n=q1 w\n\nwith probability pi.\n\n3.2 Main Result\nWe now state our result that achieves both the guarantees of De\ufb01nition 2.\nTheorem 3 (Private spectral sparsi\ufb01cation). Given privacy parameter (\u21b5, ), accuracy parameter\n\", an n vertex graph G, let Ln denote the Laplacian of an unweighted complete graph, Kn, and\nw = O\u21e3 log(1/)\n\n\u21b5\" pn log n log(1/)\u2318. Then we have the following:\n\n1. PRIVATE-SPARSIFY is a polynomial time (\u21b5, )-differentially private algorithm.\n\n).\n\n\u21b52\n\ni\n\n2. eG PRIVATE-SPARSIFY(G,\", (\u21b5, )) has O(n/\"2) edges, such that with probability at\n\nleast 9/10, we have\n\nProof Sketch of Theorem 3. Our algorithm requires performing matrix computations and solving a\nsemi-de\ufb01nite program. All matrix computation requires at most O(n3) time. Solving a semi-de\ufb01nite\nprogram takes poly(n) time, where the exact polynomial depends on whether we use interior point,\nellipsoid method, or primal-dual approach. To prove privacy, we need to argue that computing L\nand R is (\u21b5/3,/ 3)-differentially private. Blocki et al. [9] showed that computing R is (\u21b5/3,/ 3)-\ndifferentially private while computing G0 is private due to Gaussian mechanism. The privacy guarantee\non L follows using arguments similar to [9].\nFor the accuracy proof, recall that we approximate the left singular space by L and the right singular\nspace by R by using random Gaussian matrix, and then use (L(R>R)\u2020L>)ii to approximate effective\n\nIt is straightforward, then, to argue that sampling using the effective resistance thus computed would\nresult in a graph. Proving that the estimated effective resistances provide the spectral approximation\nis a bit more involved and relies on matrix Bernstein inequality [53].\n\nresistancee\u2327i (see Step 2) \u2013 this follows from the concentration property of random Gaussian matrices.\nOur proof requires the spectral relation between \u00afG and bG since Xi is de\ufb01ned with respect to the edges\nof \u00afG ande\u2327i is de\ufb01ned with respect to bG. Since the solution of SDP-1 is = cq log n log(1/)\n\n, we have\n\nn\u21b52\n\nLeG0 cr log n log(1/)\n\nn\u21b52\n\nLn L \u00afG LeG0 + cr log n log(1/)\n\nn\u21b52\n\nLn.\n\n5\n\n(1 \")\u21e3\u21e31 \n\nw\n\nn\u2318 LG +\n\nw\nn\n\nLn\u2318 LeG (1 + \")\u21e3\u21e31 \n\nw\n\nn\u2318 LG +\n\nw\nn\n\nLn\u2318 .\n\n\fAnother application of matrix Bernstein inequality gives us for some small constant \u21e2> 0,\n\nLn# 1 .\nLet ei be the edges in \u00afG de\ufb01ned in Algorithm 1. De\ufb01ne random variables Xi as follow:\n\n+ \u21e2r log n log(1/)\n\nLn LeG0 LbG\n\nPr\"LbG \u21e2r log n log(1/)\nXi :=( eie>i\nLet Y =Pi Xi. Then we show the following:\n\nwith probability pi\nwith probability 1 pi\nand Xi O\u2713\n\nE[Xi] = L \u00afG\n\nE[Y ] =Xi\nPr [(1 \")L \u00afG Y (1 + \")L \u00afG] 1 nec\"2 log(n/)/3\n\nc log(n/)\u25c6 L \u00afG.\n\nApplying matrix Bernstein inequality [53] gives\n\npi\n\n0\n\nn\u21b52\n\n.\n\n\"2\n\nn\u21b52\n\nDE \u00afG = Y implies (1 \")L \u00afG E>\u00afG\nDE \u00afG (1 + \")L \u00afG with probability\nDE \u00afG can be the Laplacian of a dense graph. Using the result of Lee and\nhas O(n/\"2) edges. Combining all\nand that LeG\n\nfor large enough c. Now E>\u00afG\nat least 99/100. Now E>\u00afG\nSun [35], we have (1 \")LeG L \u00afG (1 + \")LeG\nthese partial orderings gives us the accuracy bound.\nExtension to Weighted Graphs We can use a standard technique to extend the result in Theorem 3\nto weighted graphs. We assume that the weights on the graph are integers in the range [1, poly n].\nWe consider different levels (1 + \")i for i 2 [c log n] for some constant c. Then, we consider input\ni=1 LG,i, where LG,i has edges with weights0, (1 + \")i .\ngraphs of the following form, LG =Pc log n\nIn other words, we use the (1 + \")-ary representation of weights on the edges and partition the graph\naccordingly and run Algorithm 1 on each LG,i. Again, since there are at most poly log n levels, the\n\"2 ). Since eG is (\u21b5 poly log n, poly log n)-differentially private, we can\nnumber of edges in eG is eO( n\nrun another instance of [35] to get a graph bG with O( n\n\nPrivate Estimation of Effective Resistances. Drineas and Mahoney showed a close relation between\neffective resistance and statistical leverage scores [16]. Their result imply that effective resistance are\nimportant statistical property of a graph. As a by-product of Algorithm 1, we can privately estimate\nthe effective resitances of the edges. As mentioned earlier, this is not possible through previous\napproaches. For example, both Blocki et al. [9] and Dwork et al. [19] only approximates the right\nsingular space, which is not suf\ufb01cient to capture enough statistics to estimate the effective resistances.\nWhile effective resistance have been less explored, statistical leverage scores have been widely used\nby statisticians. Consequently over the period of time, researchers have explored some other statistical\napplications includes random feature learning [47] and quadrature [6] of effective resistances.\n\n\"2 ) edges.\n\n4 Applications of Theorem 3\n\nSince differential privacy is preserved under any post-processing, the output of Algorithm 1 can be\nused in many graph problems. It is well known that the spectrum of a graph de\ufb01nes many structural\nand combinatorial properties of the graph [20]. Since Theorem 3 guarantees a sparse graph, it\nallows us to signi\ufb01cantly improve the run-time for many graph algorithms, for example, min-cuts and\nseparators [49], heat kernel computations [41], approximate Lipschitz learning on graphs [32], spectral\nclustering, linear programming [34], solving Laplacian systems [33], approximation algorithms [29,\n42], and matrix polynomials in the graph Laplacian [12].\nFor example, Theorem 3 with Goemans and Williamson [21] (and Arora et al. [5]) allows us to\noutput a partition of nodes that approximates MAX-CUT (SPARSEST-CUT, respectively). We can\nalso answer all possible cut queries with the same accuracy as previous best in O(|S|) time instead of\nO(n|S|) time. This is a signi\ufb01cant improvement for practical graphs. Lastly, we exhibit the versatility\nof our result by showing its application in extracting low-dimensional representations when data arise\nfrom sampling a probability distribution on a manifold, a typical task in representation learning.\n\n6\n\n\fAdditive Error\n\neO\u21e3 |S|pn\n\u21b5 \u2318\neO\u21e3 |S||T|\n\u21b5pn\u2318\neO\u21e3 |Sm|pn\n\u2318\neO(\neO\u21e3 pn\n\u21b5 \u2318\n\n1pn\u21b52 )\n\n\u21b5\n\n\u21b5\n\nPrevious Best Error\n\n\u21b5 \u2318 [9, 19]\neO\u21e3 |S|pn\neO\u2713pn|S||T|\n\u25c6 [23]\n\u21b5 \u2318 [23]\neO\u21e3 n3/2\npn\neO(\n|Ss|\u21b5 ) [23]\neO\u21e3 pn\n\u21b5 \u2318 [19]\n\nTheorem\nTheorem 4\n\nTheorem 4\n\nTheorem 5\nTheorem 5\nTheorem 6\n\nProblem\n\n(S, V \\S) queries\n(S, T ) queries\n\nMAX-CUT\n\nSPARSEST-CUT\n\nEigenmap\n\nTable 1: Applications of Our Results for =\u21e5( n log n) (\u21b5> 0 is an arbitrary constant, Sm is vertex set\ninducing MAX-CUT, Sc is vertex set inducing SPARSEST-CUT, Q is vertex set for cut query).\n\n\u21b5\"\n\nIn the rest of this section, we discuss these applications in more detail (see Table 1 for comparison).\nFor this section, let w = O( log(1/)\nAnswering (S, T ) cut queries ef\ufb01ciently. Cut queries is one of the most widely studied problem in\nprivate graph analysis [9, 23, 54] and has wide applications [43, 46]. Given a graph G, an (S, T )-cut\nquery requires us to estimate the sum of weights of edges between the vertex sets in S and T . Since\nour output is a graph, we can use it to answer (S, T ) cut queries privately and more ef\ufb01ciently.\nTheorem 4 (Cut queries). Given a graph with n vertices, there is an ef\ufb01cient (\u21b5, )-differentially\n\n\u21b5\" pn log n log(1/)).\n\nn\nincur an additive error O( min{|S|,|V \\S|}\n\nprivate algorithm that outputs a graph eG, which can be used to answer all possible (S, T )-cut queries\nwith additive error O(|S||T|\u21b5\" q log n log3(1/)\n). In particular, when T = V \\S, then our algorihtm\nlog3(1/)pn log n log(1/)).\n\u21b5pn|S||T|) additive error. Note that\nThis improves the result of Gupta et al. [23] who incur an O( 1\nBlocki et al. [9] and Dwork et al. [19] can be used to only answer cut queries when T = V \\S. For\n(S, V \\S)-cut queries, recall that Dwork et al. output C = E>\nG EG + N, where N is a Gaussian matrix\nwith appropriate noise to preserve differential privacy. For a set of q cut queries of form (S, V \\S)\nwith |S|\uf8ff| V \\S|, standard concentration result of Gaussian distribution implies that Dwork et\nal. [19] incur |1>S N 1S| = O(|S|plog(1/) log(q)/\u21b5) additive error, where 1S 2{ 0, 1}n has 1 only\nin coordinate corresponding to S \u2713 [n]. In particular, if we wish to answer all possible cut queries,\nit leads to an additive error O(|S|pn log(1/)/\u21b5). We thus match these bounds [9, 19, 54] while\nanswering an (S, V \\S) query in O(|S|/\"2) amortized time instead of O(|S|2) amortized time.\nOptimization problems. Given a graph G = (V, E) on a vertex set V , the goal of MAX-CUT to\noutput a set of vertices S \u2713 V that maximizes the value of S(G). It is well known that solving\nMAX-CUT exactly and even with (0.87856 + \u21e2)-approximation for \u21e2> 0 is NP-hard [31]. However,\nGoemans and Williamson [21] gave an elegant polynomial time algorithm for computing 0.87856 \u2318\napproximation to MAX-CUT for some \u2318> 0, thereby giving an approximation algorithm that is\noptimal with respect to the multiplicative approximation. Another problem that is considered in graph\ntheory is the problem of \ufb01nding SPARSEST-CUT. Here, given a graph G = (V, E) on a vertex set\nS (G)\nV , the goal is to output a set of vertices S \u2713 V that minimizes the value\n|S|(n|S|). The proposed\nalgorithms for these problems \ufb01rst computes a private sparse graph as in Algorithm 1 followed by a\nrun of the non-private algorithm ([21] in the case of MAX-CUT and [5] in the case of SPARSEST-CUT)\nto obtain a set of vertices S. We show the following guarantee.\nTheorem 5 (Optimization problems). There is a polynomial-time algorithm that, for an n-vertex\ngraph G := (V, E), is (\u21b5, )-differentially private with respect to the edge level privacy and produces\na partition of nodes (S, V \\S) satisfying\n\nMAX-CUT: S(G) (0.87856 \u2318)\u2713 1 \"\n\n1 + \"\u25c6 max\n\nS\u2713V\n\nS(G)O\u2713 w|S|\n\u21b5 \u25c6 .\n\nThere is a polynomial-time algorithm that, for an n-vertex graph G := (V, E), is (\u21b5, )-differentially\nprivate with respect to the edge level privacy and produces a partition of nodes (S, V \\S) satisfying\n|S|(n | S|)\u25c6 + O\u2713 w log2 n\n\u25c6 .\nSPARSEST-CUT:\n\n|S|(n | S|) \uf8ff O(plog n)\u2713 1 + \"\n\nS\u2713V\u2713 S(G)\n\n1 \"\u25c6 min\n\nS(G)\n\nn\n\n7\n\n\fpn\n\"|S|\n\n) for SPARSEST-CUT.\n\n\u21b5 ) for MAX-CUT and O(\n\nThe above theorem states that we can approximately and privately compute MAX-CUT and SPARSEST-\nCUT of an arbitrary graph in polynomial time. Further, the price of privacy in the form of the additive\nerror scales sublinearly with n. On the other hand, if we use the privatized graph of [23], it would\nincur an error of O( n3/2\nOne may argue that we can use the output of Dwork et al. [19] or Blocki et al. [9] to solve theses\noptimization problem. Unfortunately, it is not the case. To see this, let us recall their output. For a\ngiven graph G, Blocki et al. [9] output R>R, where R is as computed in Step 4 of Algorithm 1. On\nthe other hand, Dwork et al. [19] computes LG + N, where N is a symmetric Gaussian matrix with\nentries sampled i.i.d. with variance required to preserve privacy.\nBoth these approach output a symmetric matrix; however, the output of Dwork et al. [19] is neither a\nLaplacian nor a positive semi-de\ufb01nite matrix, a requirement in all the existing algorithms. On the\nother hand, even though the output of Blocki et al. [9] is a positive semi-de\ufb01nite matrix, it can have\npositive off-diagonal entries. As such, we cannot use them in the existing algorithms for MAX-CUT\nor SPARSEST-CUT since (analysis of) existing techniques for these optimization problems requires\ngraph to be positively weighted (see the note by Har-Peled [24])4. Even if we can use their output,\nour algorithm allows a faster computation since we signi\ufb01cantly reduce the number of constraints in\nthe corresponding semi-de\ufb01nite programs for these optimization problems.\nLearning laplacian eigenmap. A basic challenge in machine learning is to learn a low-dimensional\nrepresentation of data drawn from a probability distribution on a manifold. This is also referred to as\nthe problem of manifold learning. In recent years, many approaches have been proposed for manifold\nlearning, including that of ISOMAP, local linear embedding, and Laplacian eigenmap.\nIn particular, the state-of-the-art Laplacian eigenmaps are relatively insensitive to outliers and noise\ndue to locality-preserving character. In this approach, given n data samples {x1, . . . , xn}2 Rd, we\nconstruct a weighted graph G with n nodes and a set of edges connecting neighboring points. The\nembedding map is now provided by computing the top k eigenvectors of the graph Laplacian. There\nare multiple ways in which we assign an edge e = (u, v) and edge weight between nodes u and v. We\nconsider the following neighborhood graph: we have an edge e = (u, v) if kxu xvk2 \uf8ff \u21e2 for some\nparameter \u21e2. If there is an edge, then that edge is given a weight as per the heat kernel, ekxuxvk2\nfor some parameter t 2 R. The goal here is to \ufb01nd the embedding map, i.e., an orthonormal projection\nmatrix, U U>, close to the optimal projection matrix, UkU>k , where the columns of Uk are the top-k\neigenvectors of LG. Using our framework, we can guarantee the following for privately learning the\nLaplacian eigenmap matching the bound achieved by previous results [19].\nTheorem 6 (Laplacian eigenmap). Let G be the neighborhood graph formed by the data samples\n{x1,\u00b7\u00b7\u00b7 , xn}2 Rd as described above. Let LG,k = UkU>k LG, where the columns of Uk are the\ntop-k eigenvectors of LG. Then, there is an ef\ufb01cient (\u21b5, )-differentially private learning algorithm\nthat outputs a rank-k orthonormal matrix U 2 Rn\u21e5k such that\n\n2/t\n\n(I U U>)LG2 \uf8ff (1 + \")kLG LG,kk2 + O (w) .\n\n5 Differentially Private Cut Sparsi\ufb01cation\n\nBenzur and Karger [8] introduced the notion of cut sparsi\ufb01cation. In this section, we present an\nalgorithm that outputs a cut sparsi\ufb01er while preserving (\u21b5, 0)-differential privacy. We use this\nalgorithm to answer cut queries, approximately computing MAX-CUT, SPARSEST-CUT, and EDGE-\nEXPANSION, with (\u21b5, 0)-differential privacy. We show the following:\nTheorem 7. Let G = (V, E) be an unweighted graph. Given an approximation parameter 0 <\"< 1\nand privacy parameter \u21b5, PRIVATE-CUT-SPARSIFY, described in Algorithm 2, is (\u21b5, 0)-differentially\nprivate. Further, PRIVATE-CUT-SPARSIFY outputs an n vertices eO(n/\"2) edges graph, eG, such that,\nwith probability at least 99/100, we have 8S \u2713 V,\n1S + O\u2713qn1>S Ln1S\u25c6.\n(1 \")1>S LeG\n\n1S O\u2713qn1>S Ln1S\u25c6 \uf8ff \u21b5 \u00b7 1>S LG1S \uf8ff (1 + \")1>S LeG\n\n4Alon et al. [3] gave an algorithm for solving MAX-CUT for real-weight graphs using quadratic program (in-\nstead of semi-de\ufb01nite program based approach [21]), but that leads to an O(log n) multiplicative approximation.\n\n8\n\n\f.\n\nAlgorithm 2 PRIVATE-CUT-SPARSIFY (G,\",\u21b5 )\nInput: An n vertex graph G = (VG, EG), privacy parameters (\u21b5, ), approximation parameter \".\nOutput: A Laplacian LeG\n1: Privatize. Construct a complete graph bG with weights we de\ufb01ned as follows:\nif e 2 EG\nif e 2 EG\nif e /2 EG\nif e /2 EG\n\n+1 with probability (1 + \u21b5)/2\n1 with probability (1 \u21b5)/2\n+1 with probability 1/2\n1 with probability 1/2\n\n.\n\nwe :=8>><>>:\n\n2: Compute \u00afG using the linear program of Gupta et al. [23] on bG.\n3: Output eG by running the algorithm of Benzur and Karger [8] on \u00afG.\nThe above theorem says that any cut can be approximated by our cut sparsi\ufb01er up to eO(pn(n s)s)\n\nerror, while preserving (\u21b5, 0)-differential privacy. This is an instance based bound and matches the\naccuracy achieved by our graph sparsi\ufb01cation result (and the best possible [9, 54]) when s = O(n).\n\n6 Discussion\n\nIn this paper, we introduced private spectral sparsi\ufb01cation of graphs. We gave ef\ufb01cient algorithms\nto compute one such sparsi\ufb01cation. Our algorithm outputs a graph with O(n/\"2) edges, while\npreserving differential privacy. Our algorithmic framework allows us to compute an approximation to\nMAX-CUT and SPARSEST-CUT with better accuracy than previously known, and a \ufb01rst algorithm for\nlearning differentially private Laplacian eigenmaps.\nOur algorithm uses both importance sampling and random sketching. At a high level, sketching\nallows us to ensure privacy and importance sampling allows us to produce spectral sparsi\ufb01cation. To\nthe best of our knowledge, this is the \ufb01rst instance of using importance sampling in the context of\nprivacy. Since important sampling is an important tool in non-private low-space algorithms [55], this\nwe believe can lead to development of other private algorithms.\nOur work differs from previous works that use random projections or graph sparsi\ufb01cation [9, 54]. The\nonly work that we are aware of which uses spectral sparsi\ufb01cation in the context of differential privacy\nis that of Upadhyay [54] with an aim to improve the run-time ef\ufb01ciency of Blocki et al. [9]; however,\ntheir algorithm does not output a graph let alone a sparse graph. Hence, their approach cannot be\nused to approximate MAX-CUT or SPARSE-CUT \u2013 the only method to solve these problems would\nbe to run all possible cut queries leading to an error of O(n3/2/\u21b5). This follows because the error\nper cut query would be O(\n), and since there are 2n possible cuts, an application of Chernoff\nbound results in worst case error O( n3/2\nOn the other hand, Gupta et al. [23] give an algorithm to output a graph that preserves cut functions\non a graph. However, their output does not preserve the spectral properties of the graph and so cannot\nbe used in spectral applications of graphs, such as Laplacian eigenmap or learning Lipschitz functions\non graphs. Moreover, their algorithm incurs large additive error. We also give a tighter analysis for a\nvariant of their algorithm to achieve an instance based additive error.\nAcknowledgements. This research was supported, in part, by NSF BIGDATA grants IIS-1546482\nand IIS-1838139, and DARPA award W911NF1820267. This work was done when Jalaj Upadhyay\nwas working as a postdoctoral researcher with Raman Arora at the Johns Hopkins University. Authors\nwould like to thank Adam Smith, Lorenzo Orecchia, Cynthia Steinhardt, and Sarvagya Upadhyay for\ninsightful discussions during the early stages of the project.\n\n\u21b5 ), matching the guarantee of Gupta et al. [23].\n\npn|S|\"\n\nReferences\n[1] Dimitris Achlioptas and Frank McSherry. Fast computation of low-rank matrix approximations. Journal of\n\nthe ACM (JACM), 54(2):9, 2007.\n\n9\n\n\f[2] Zeyuan Allen-Zhu, Zhenyu Liao, and Lorenzo Orecchia. Spectral sparsi\ufb01cation and regret minimization\n\nbeyond matrix multiplicative updates. In STOC, pages 237\u2013245. ACM, 2015.\n\n[3] Noga Alon, Konstantin Makarychev, Yury Makarychev, and Assaf Naor. Quadratic forms on graphs,.\n\nInventiones Mathematicae, 163(3):499\u2013522, 2006.\n\n[4] Raman Arora, Vladimir Braverman, and Jalaj Upadhyay. Differentially private robust low-rank approxima-\n\ntion. In Advances in Neural Information Processing Systems, pages 4137\u20134145, 2018.\n\n[5] Sanjeev Arora, Satish Rao, and Umesh Vazirani. Expander \ufb02ows, geometric embeddings and graph\n\npartitioning. Journal of the ACM (JACM), 56(2):5, 2009.\n\n[6] Francis Bach. On the equivalence between kernel quadrature rules and random feature expansions. The\n\nJournal of Machine Learning Research, 18(1):714\u2013751, 2017.\n\n[7] Joshua Batson, Daniel A Spielman, and Nikhil Srivastava. Twice-ramanujan sparsi\ufb01ers. SIAM Journal on\n\nComputing, 41(6):1704\u20131721, 2012.\n\n[8] Andr\u00e1s A Bencz\u00far and David R Karger. Approximating st minimum cuts in \u00f5 (n 2) time. In STOC, pages\n\n47\u201355. ACM, 1996.\n\n[9] Jeremiah Blocki, Avrim Blum, Anupam Datta, and Or Sheffet. The Johnson-Lindenstrauss Transform\n\nItself Preserves Differential Privacy. In FOCS, pages 410\u2013419, 2012.\n\n[10] Jeremiah Blocki, Avrim Blum, Anupam Datta, and Or Sheffet. Differentially private data analysis of social\n\nnetworks via restricted sensitivity. In ITCS, pages 87\u201396, 2013.\n\n[11] Shixi Chen and Shuigeng Zhou. Recursive mechanism: towards node differential privacy and unrestricted\njoins. In Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data, pages\n653\u2013664. ACM, 2013.\n\n[12] Dehua Cheng, Yu Cheng, Yan Liu, Richard Peng, and Shang-Hua Teng. Ef\ufb01cient sampling for gaussian\n\ngraphical models via spectral sparsi\ufb01cation. In COLT, pages 364\u2013390, 2015.\n\n[13] Michael B. Cohen, Yin Tat Lee, Cameron Musco, Christopher Musco, Richard Peng, and Aaron Sidford.\n\nUniform sampling for matrix approximation. In ITCS, pages 181\u2013190. ACM, 2015.\n\n[14] Michael B Cohen, Cameron Musco, and Christopher Musco. Input sparsity time low-rank approximation\nvia ridge leverage score sampling. In Proceedings of the Twenty-Eighth Annual ACM-SIAM Symposium on\nDiscrete Algorithms, pages 1758\u20131777. SIAM, 2017.\n\n[15] James C Costello, Mehmet M Dalkilic, Scott M Beason, Jeff R Gehlhausen, Rupali Patwardhan, Sumit\nMiddha, Brian D Eads, and Justen R Andrews. Gene networks in drosophila melanogaster: integrating\nexperimental data to predict gene function. Genome biology, 10(9):R97, 2009.\n\n[16] Petros Drineas and Michael W Mahoney. Effective resistances, statistical leverage, and applications to\n\nlinear equation solving. arXiv preprint arXiv:1005.3097, 2010.\n\n[17] Cynthia Dwork and Aaron Roth. The algorithmic foundations of differential privacy. Foundations and\n\nTrends in Theoretical Computer Science, 9(3-4):211\u2013407, 2014.\n\n[18] Cynthia Dwork, Guy N. Rothblum, and Salil P. Vadhan. Boosting and Differential Privacy. In FOCS,\n\npages 51\u201360, 2010.\n\n[19] Cynthia Dwork, Kunal Talwar, Abhradeep Thakurta, and Li Zhang. Analyze Gauss: Optimal Bounds for\n\nPrivacy-Preserving Principal Component Analysis. In STOC, pages 11\u201320, 2014.\n\n[20] Miroslav Fiedler. Algebraic connectivity of graphs. Czechoslovak mathematical journal, 23(2):298\u2013305,\n\n1973.\n\n[21] Michel X Goemans and David P Williamson. Improved approximation algorithms for maximum cut and\nsatis\ufb01ability problems using semide\ufb01nite programming. Journal of the ACM (JACM), 42(6):1115\u20131145,\n1995.\n\n[22] Anupam Gupta, Katrina Ligett, Frank McSherry, Aaron Roth, and Kunal Talwar. Differentially private\n\ncombinatorial optimization. In SODA, pages 1106\u20131125. SIAM, 2010.\n\n[23] Anupam Gupta, Aaron Roth, and Jonathan Ullman. Iterative constructions and private data release. In\n\nTCC, pages 339\u2013356. Springer, 2012.\n\n10\n\n\f[24] Sariel Har-Peled. Approximate max-cut. 2005.\n\n[25] Moritz Hardt and Aaron Roth. Beating randomized response on incoherent matrices. In Howard J. Karloff\n\nand Toniann Pitassi, editors, STOC, pages 1255\u20131268. ACM, 2012.\n\n[26] Minoru Kanehisa and Susumu Goto. Kegg: kyoto encyclopedia of genes and genomes. Nucleic acids\n\nresearch, 28(1):27\u201330, 2000.\n\n[27] Vishesh Karwa, Sofya Raskhodnikova, Adam Smith, and Grigory Yaroslavtsev. Private analysis of graph\n\nstructure. Proceedings of the VLDB Endowment, 4(11):1146\u20131157, 2011.\n\n[28] Shiva Prasad Kasiviswanathan, Kobbi Nissim, Sofya Raskhodnikova, and Adam Smith. Analyzing graphs\n\nwith node differential privacy. In Theory of Cryptography, pages 457\u2013476. Springer, 2013.\n\n[29] Jonathan A Kelner, Yin Tat Lee, Lorenzo Orecchia, and Aaron Sidford. An almost-linear-time algorithm\nfor approximate max \ufb02ow in undirected graphs, and its multicommodity generalizations. In SODA, pages\n217\u2013226. SIAM, 2014.\n\n[30] Subhash Khot. On the power of unique 2-prover 1-round games. In Proceedings of the thiry-fourth annual\n\nACM symposium on Theory of computing, pages 767\u2013775. ACM, 2002.\n\n[31] Subhash Khot, Guy Kindler, Elchanan Mossel, and Ryan O\u2019Donnell. Optimal inapproximability results\n\nfor max-cut and other 2-variable csps? SIAM Journal on Computing, 37(1):319\u2013357, 2007.\n\n[32] Rasmus Kyng, Anup Rao, Sushant Sachdeva, and Daniel A Spielman. Algorithms for lipschitz learning on\n\ngraphs. In Conference on Learning Theory, pages 1190\u20131223, 2015.\n\n[33] Yin Tat Lee, Richard Peng, and Daniel A Spielman. Sparsi\ufb01ed cholesky solvers for sdd linear systems.\n\narXiv preprint arXiv:1506.08204, 2015.\n\n[34] Yin Tat Lee and Aaron Sidford. Path \ufb01nding methods for linear programming: Solving linear programs in\n\no (vrank) iterations and faster algorithms for maximum \ufb02ow. In FOCS, pages 424\u2013433. IEEE, 2014.\n\n[35] Yin Tat Lee and He Sun. Constructing linear-sized spectral sparsi\ufb01cation in almost-linear time. In FOCS,\n\npages 250\u2013269. IEEE, 2015.\n\n[36] Yin Tat Lee and He Sun. An sdp-based algorithm for linear-sized spectral sparsi\ufb01cation. In STOC, pages\n\n678\u2013687. ACM, 2017.\n\n[37] Adam Marcus, Daniel A Spielman, and Nikhil Srivastava. Interlacing families ii: Mixed characteristic\n\npolynomials and the kadison-singer problem. arXiv preprint arXiv:1306.3969, 2013.\n\n[38] Carl D Meyer, Jr. Generalized inversion of modi\ufb01ed matrices. SIAM Journal on Applied Mathematics,\n\n24(3):315\u2013323, 1973.\n\n[39] Darakhshan Mir and Rebecca N Wright. A differentially private estimator for the stochastic kronecker\n\ngraph model. In Proceedings of the 2012 Joint EDBT/ICDT Workshops, pages 167\u2013176. ACM, 2012.\n\n[40] Kobbi Nissim, Sofya Raskhodnikova, and Adam Smith. Smooth sensitivity and sampling in private data\n\nanalysis. In STOC, 2007.\n\n[41] Lorenzo Orecchia, Sushant Sachdeva, and Nisheeth K Vishnoi. Approximating the exponential, the lanczos\nmethod and an o (m)-time spectral algorithm for balanced separator. In STOC, pages 1141\u20131160. ACM,\n2012.\n\n[42] Richard Peng. Approximate undirected maximum \ufb02ows in o(m polylog (n)) time.\n\n1862\u20131867. Society for Industrial and Applied Mathematics, 2016.\n\nIn SODA, pages\n\n[43] J Scott Provan and Michael O Ball. The complexity of counting cuts and of computing the probability that\n\na graph is connected. SIAM Journal on Computing, 12(4):777\u2013788, 1983.\n\n[44] Sofya Raskhodnikova and Adam D. Smith. Lipschitz extensions for node-private graph statistics and the\n\ngeneralized exponential mechanism. In FOCS, pages 495\u2013504, 2016.\n\n[45] Vibhor Rastogi, Michael Hay, Gerome Miklau, and Dan Suciu. Relationship privacy: output perturbation\n\nfor queries with joins. In Symp. Principles of Database Systems (PODS), pages 107\u2013116, 2009.\n\n[46] Carsten Rother, Vladimir Kolmogorov, and Andrew Blake. Grabcut: Interactive foreground extraction\nusing iterated graph cuts. In ACM transactions on graphics (TOG), volume 23, pages 309\u2013314. ACM,\n2004.\n\n11\n\n\f[47] Alessandro Rudi and Lorenzo Rosasco. Generalization properties of learning with random features. In\n\nAdvances in Neural Information Processing Systems, pages 3215\u20133225, 2017.\n\n[48] Tamas Sarlos. Improved approximation algorithms for large matrices via random projections. In FOCS,\n\npages 143\u2013152. IEEE, 2006.\n\n[49] Jonah Sherman. Nearly maximum \ufb02ows in nearly linear time. In FOCS, pages 263\u2013269. IEEE, 2013.\n\n[50] Daniel A Spielman and Nikhil Srivastava. Graph sparsi\ufb01cation by effective resistances. SIAM Journal on\n\nComputing, 40(6):1913\u20131926, 2011.\n\n[51] Daniel A Spielman and Nikhil Srivastava. An elementary proof of the restricted invertibility theorem.\n\nIsrael Journal of Mathematics, 190(1):83\u201391, 2012.\n\n[52] Daniel A Spielman and Shang-Hua Teng. Spectral sparsi\ufb01cation of graphs. SIAM Journal on Computing,\n\n40(4):981\u20131025, 2011.\n\n[53] Joel A Tropp. User-friendly tail bounds for sums of random matrices. Foundations of computational\n\nmathematics, 12(4):389\u2013434, 2012.\n\n[54] Jalaj Upadhyay. Random Projections, Graph Sparsi\ufb01cation, and Differential Privacy. In ASIACRYPT (1),\n\npages 276\u2013295, 2013.\n\n[55] David P. Woodruff. Sketching as a tool for numerical linear algebra. Foundations and Trends in Theoretical\n\nComputer Science, 10(1-2):1\u2013157, 2014.\n\n12\n\n\f", "award": [], "sourceid": 7360, "authors": [{"given_name": "Raman", "family_name": "Arora", "institution": "Johns Hopkins University"}, {"given_name": "Jalaj", "family_name": "Upadhyay", "institution": "Apple"}]}