{"title": "On Robustness to Adversarial Examples and Polynomial Optimization", "book": "Advances in Neural Information Processing Systems", "page_first": 13760, "page_last": 13770, "abstract": "We study the design of computationally efficient algorithms with provable guarantees, that are robust to adversarial (test time) perturbations. While there has been an explosion of recent work on this topic due to its connections to test time robustness of deep networks, there is limited theoretical understanding of several basic questions like (i) when and how can one design provably robust learning algorithms? (ii) what is the price of achieving robustness to adversarial examples in a computationally efficient manner?\n\nThe main contribution of this work is to exhibit a strong connection between achieving robustness to adversarial examples, and a rich class of polynomial optimization problems, thereby making progress on the above questions. In particular, we leverage this connection to (a) design computationally efficient robust algorithms with provable guarantees for a large class of hypothesis, namely linear classifiers and degree-2 polynomial threshold functions~(PTFs), (b) give a precise characterization of the price of achieving robustness in a computationally efficient manner for these classes, (c) design efficient algorithms to certify robustness and generate adversarial attacks in a principled manner for 2-layer neural networks. We empirically demonstrate the effectiveness of these attacks on real data.", "full_text": "On Robustness to Adversarial Examples and\n\nPolynomial Optimization\n\nDepartment of Computer Science\n\nPranjal Awasthi\nRutgers University\n\npranjal.awasthi@rutgers.edu\n\nAbhratanu Dutta\n\nDepartment of Computer Science\n\nNorthwestern University\n\nabhratanudutta2020@u.northwestern.edu\n\nAravindan Vijayaraghavan\nDepartment of Computer Science\n\nNorthwestern University\n\naravindv@northwestern.edu\n\nAbstract\n\nWe study the design of computationally ecient algorithms with provable\nguarantees, that are robust to adversarial (test time) perturbations. While\nthere has been an explosion of recent work on this topic due to its connec-\ntions to test time robustness of deep networks, there is limited theoretical\nunderstanding of several basic questions like (i) when and how can one de-\nsign provably robust learning algorithms? (ii) what is the price of achieving\nrobustness to adversarial examples in a computationally ecient manner?\nThe main contribution of this work is to exhibit a strong connection between\nachieving robustness to adversarial examples, and a rich class of polynomial\noptimization problems, thereby making progress on the above questions.\nIn particular, we leverage this connection to (a) design computationally\necient robust algorithms with provable guarantees for a large class of\nhypothesis, namely linear classi\ufb01ers and degree-2 polynomial threshold\nfunctions (PTFs), (b) give a precise characterization of the price of achieving\nrobustness in a computationally ecient manner for these classes, (c) design\necient algorithms to certify robustness and generate adversarial attacks in\na principled manner for 2-layer neural networks. We empirically demonstrate\nthe eectiveness of these attacks on real data.\n\n1 Introduction\nThe empirical success of deep learning has led to numerous unexplained phenomena about\nwhich our current theoretical understanding is limited. Examples include the ability of\ncomplex models to generalize well and eectiveness of \ufb01rst order methods on optimizing\ntraining loss. The focus of this paper is on the phenomenon of adversarial robustness, that\nwas \ufb01rst pointed out by Szegedy et al. [33]. On many benchmark data sets, deep networks\noptimized on the training set can often be fooled into misclassifying a test example by making\na small adversarial perturbation that is imperceptible to a human labeler. This has led to a\nproliferation of work on designing robust algorithms that defend against such adversarial\nperturbations, as well as attacks that aim to break these defenses.\nIn this work we choose to focus on perturbation defense, the most widely studied formulation\nof adversarial robustness [24]. In the perturbation defense model, given a classi\ufb01er f, an\nadversary can take a test example x generated from the data distribution and perturb it to \u02dcx\n\n33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada.\n\n\fsuch that \u00cex \u2260 \u02dcx\u00ce \u00c6 \u201d. Here \u201d characterizes the amount of power the adversary has and the\ndistance is typically measured in the \u00b8\u0152 norm (other norms that have been studied include\nthe \u00b82 norm). Given a loss function \u00b8(\u00b7), the goal is to optimize the robust loss de\ufb01ned as\n\nE(x,y)\u2265D\u00cb max\n\n\u02dcx:\u00cex\u2260\u02dcx\u00ce\u0152\u00c6\u201d\n\n\u00b8(f(\u02dcx), y)\u00c8.\n\nOne would expect that when \u201d is small the label y of an example does not change, thereby\nmotivating the robust loss objective. Despite a recent surge in eorts to theoretically\nunderstand adversarial robustness [36, 37, 38, 21, 30, 13, 4, 10, 16, 34, 25, 26, 11], several\ncentral questions remain open. How can one design provable polynomial time algorithms\nthat are robust to adversarial perturbations? Given a classi\ufb01er and a test input, how can one\nprovably construct an adversarial example in polynomial time or certify that none exists?\nWhat computational barriers exist when designing adversarially robust learning algorithms?\nIn this work we identify and study a natural class of polynomial optimization problems\nthat are intimately connected to adversarial robustness, and help us shed new light on all\nthree of the above questions simultaneously! As a result we obtain the \ufb01rst polynomial time\nlearning algorithms for a large class of functions that are optimally robust to adversarial\nperturbations. Furthermore, we also provide nearly matching computational intractability\nresults that, together with our upper bounds give a sharp characterization of the price of\nachieving adversarial robustness in a computationally ecient manner. We now summarize\nour main results.\nOur Contributions Polynomial optimization and Adversarial Robustness. We\nidentify a natural class of polynomial optimization problems that provide a common and\nprincipled framework for studying various aspects of adversarial robustness. These problems\nare also closely related to a rich class of well-studied problems that include the Grothendi\u00ebck\nproblem and its generalizations [2, 9, 1, 22]. Given a classi\ufb01er of the form sgn(g(x)) with\ng : Rn \u00e6 R, input x, and budget \u201d> 0, the optimization problem is\n\nmax\n\nz\u0153Rn:\u00cez\u00ce\u0152\u00c6\u201d\n\ng(x + z).\n\nUsually, such problems are NP-hard and one relaxes them to \ufb01nd a \u02c6z such that g(x + \u02c6z)\ncomes as close to g(x + z\u00fa) in the objective value, where z\u00fa is the optimal solution. We\ninstead require the algorithm to output a \u02c6z such that g(x + \u02c6z) \u00d8 g(x + z\u00fa) at the cost of\nviolating the \u00b8\u0152 constraint by a factor \u201c \u00d8 1. An ecient algorithm for producing such\na \u02c6z leads to an adversarial attack (in the relaxed \u00b8\u0152 neighborhood of radius \u201c\u201d) when an\nadversarial example exists. On the other hand, if the algorithm produces no \u201az, then this\nguarantees that there is no adversarial example within the \u00b8\u0152 neighborhood of radius \u201d.\nWe then design such algorithms based on convex programming relaxations to get the \ufb01rst\nprovable polynomial time adversarial attacks when the given classi\ufb01er is a degree-1 or a\ndegree-2 polynomial threshold function (PTF).\nAlgorithms for Learning Adversarially Robust Classi\ufb01ers. Next we use the algorithm\nfor \ufb01nding adversarial examples to design polynomial time algorithms for learning robust\nclassi\ufb01ers for the class of degree-1 and degree-2 polynomial threshold functions (PTFs).\nTo incorporate robustness we introduce a parameter \u201c, that helps clarify the tradeo\nwhen computational eciency is desired. We focus on the 0/1 error and say that a\nclass F of PTFs of VC dimension  is \u201c-approximately robustly learnable if there exists\na (randomized) polynomial time algorithm that, for any given \u00c1, \u201d > 0, takes as input\npoly(, 1\n\u00c1) examples generated from a distribution and labeled by a function in F that has\nzero \u201d-robust error (realizable case), outputs a classi\ufb01er from F that has (\u201d/\u201c)-robust error\nupper bounded by \u00c1. See Section 2 for the formal de\ufb01nition. We design polynomial time\nalgorithms for degree-1 and degree-2 PTFs with \u201c = 1 and \u201c = O(\u00d4log n) respectively. Our\nnext result that we discuss below a nearly matching lower bound. Together this gives nearly\noptimal approximately robust polynomial time algorithms for learning PTFs of degree at\nmost 2.\nComputational Hardness. While our algorithm for degree-1 PTFs is optimal, i.e., has\n\u201c = 1, for degree-2 and higher PTFs, we show that one indeed has to pay a price for\ncomputational robustness. We establish this by proving that robust learning of degree-2\nPTFs is computationally hard for \u201c = o(logc n), for some constant c > 0 (see Section 5\n\n2\n\n\ffor formal statements). This is in sharp contrast to the non-robust setting (\u201d = 0), where\nthere exist polynomial time algorithms for constant degree PTFs (in the literature this is\nreferred to as proper PAC learning in the realizable setting). More importantly, our lower\nbound again leverages the connection to polynomial optimization and in fact shows that\nrobust learning of degree-2 PTFs for \u201c = o(\u00d4\u00f7approx) is NP-hard where \u00f7approx is precisely\nthe hardness of approximation factor of a well-studied combinatorial optimization problem\ncalled Quadratic Programming. Hence, any signi\ufb01cant improvement in the approximation\nfactor in our upper bound is unlikely. While our hardness result applies to algorithms that\noutput a classi\ufb01er of low error, we also prove a more robust hardness result showing that for\nlearning degree-2 and higher PTFs without any loss in the robustness parameter, i.e, \u201c = 1,\nit is computationally hard to even \ufb01nd a classi\ufb01er of any constant error in the range (0, 1\n4).\nApplication to Neural Networks. Finally, we show that the connection to polynomial\noptimization also leads to new algorithms for generating adversarial attacks on neural\nnetworks. We focus on 2-layer neural networks with ReLU activations. We show that given\na network and a test input, the problem of \ufb01nding an adversarial example can also be\nphrased as an optimization problem of the kind studied for PTFs. We design a semi-de\ufb01nite\nprogramming (SDP) based polynomial time algorithm to generate an adversarial attack for\nsuch networks and compare our attack to the state-of-the-art attack of Madry et al. [24] on\nthe MNIST data set.\nComparison to Related Work. Among the several recent and concurrent works on\nthis topic, the most relevant to our result is the work of Bubeck et al. [7, 8] that studies\ncomputational complexity of robust learning. We defer other related work to Section C. In\nthe rest of the paper, we de\ufb01ne our model formally and give an overview of our techniques in\nSection 2. We then describe the connection to polynomial optimization in Section 3 and use\nit to design robust learning algorithms in Section 4, and derive computational intractability\nresults in Section 5. In Section 6, we design adversarial attacks for 2 layer neural networks,\nfollowed by conclusions in Section 7.\n\n2 Model and Preliminaries\nWe focus on binary classi\ufb01cation, and adversarial perturbations are measured in \u00b8\u0152 norm.\nFor a vector x \u0153 Rn, we have \u00cex\u00ce\u0152 = maxi |xi|. We study robust learning of polynomial\nthreshold functions (PTFs). These are functions of the form sgn(p(x)), where p(x) is a\npolynomial in n variables over the reals. Here sgn(t) equals +1, if t \u00d8 0 and \u22601 otherwise.\nGiven y, y\u00d5 \u0153 {\u22601, 1}, we study the 0/1 loss de\ufb01ned as \u00b8(y, y\u00d5) = 1 if y \u201d= y\u00d5 and 0 otherwise.\nGiven a binary classi\ufb01er sgn(g(x)), an input x\u00fa, and a budget \u201d> 0, we say that x\u00fa + z is an\nadversarial example (for input x\u00fa) if sgn(g(x\u00fa + z)) \u201d= sgn(g(x\u00fa)) and that \u00cez\u00ce\u0152 \u00c6 \u201d. One\ncould similarly de\ufb01ne the notion of adversarial examples for other norms. For a classi\ufb01er\nwith multiple outputs, we say that x\u00fa + z is an adversarial example i the largest co-ordinate\nof g(x\u00fa + z) diers from the largest co-ordinate of g(x\u00fa). We now de\ufb01ne the notion of robust\nerror of a classi\ufb01er.\nDe\ufb01nition 2.1 (\u201d-robust error). Let f(x) be a Boolean function mapping Rn to {\u22601, 1}.\nLet D be a distribution over Rn \u25ca {\u22601, 1}. Given \u201d> 0, we de\ufb01ne the \u201d-robust error of\nf with respect to D as err\u201d,D(f) = E(x,y)\u2265D# supz\u0153Bn\n\u0152(0,\u201d )\ndenotes the \u00b8\u0152 ball of radius \u201d, i.e., Bn\nAnalogous to empirical error in PAC learning, we denote \u02c6err\u201d,S(f) to be the \u201d-robust empirical\nerror of f, i.e., the robust error computed on the given sample S. To bound generalization gap,\nwe will use the notion of adversarial VC dimension as introduced in [10] (See Appendix A).\nNext we de\ufb01ne robust learning for PTFs.\nDe\ufb01nition 2.2 (\u201c-approximately robust learning). Let F be the class of degree-d PTFs\nfrom Rn \u2018\u00e6 {\u22601, 1} of VC dimension = O(nd). For \u201c \u00d8 1, an algorithm A \u201c-approximately\nrobustly learns F if the following holds for any \u00c1, \u201d, \u00f7 > 0: Given m = poly(, 1\n\u00f7) samples\nfrom a distribution D over Rn\u25ca{\u22601, 1}, if F contains a function f\u00fa such that err\u201d,D(f\u00fa) = 0,\nthen with probability at least 1\u2260 \u00f7, A runs in time polynomial in m and outputs f \u0153F such\nthat err\u201d/\u201c,D(f) \u00c6 \u00c1. If F admits such an algorithm then we say that F is \u201c-approximately\n\n(0,\u201d) \u00b8(f(x + z), y)$. Here Bn\n\n\u00c1 , 1\n\n\u0152(0,\u201d ) = {x \u0153 Rn : \u00cex\u00ce\u0152 \u00c6 \u201d}.\n\n\u0152\n\n3\n\n\frobustly learnable. Here \u201c quanti\ufb01es the price of achieving computationally ecient robust\nlearning, with \u201c = 1 implying optimal learnability.\n\nA Note about the Model and the Realizability Assumption Our de\ufb01nition of an\nadversarial example requires that sgn(g(x\u00fa + z)) \u201d= sgn(g(x\u00fa)), whereas for robust learning\nwe require a classi\ufb01er that satis\ufb01es sgn(g(x\u00fa + z)) \u201d= y, where y is the given label of x\u00fa. This\nmight create two sources of confusion to the reader: a) In general the two requirements\nmight be incompatible, and b) It might happen that initially sgn(g(x\u00fa)) predicts the true\nlabel incorrectly but there is a perturbation z such that sgn(g(x\u00fa + z)) predicts the true\nlabel correctly. In this case one should not count z as an adversarial example. To address\n(a) we would like to stress that all our guarantees hold under the realizability assumption,\ni.e., we assume that there is true function c\u00fa such that for all examples x in the support of\nthe distribution and all perturbations of magnitude upto \u201d, sgn(c\u00fa(x\u00fa + z)) = sgn(c\u00fa(x\u00fa)).\nHence, there will indeed be a target concept for which no adversarial example exists and as a\nresult will have zero robust error. To address (b) we would like to point out that in Section 4\nwhere we use the subroutine for \ufb01nding adversarial examples to learn a good classi\ufb01er sgn(g),\nwe always enforce the constraint that on the training set sgn(g(x\u00fa)) = sgn(c\u00fa(x\u00fa)) and g is as\nrobust as possible. Hence when we \ufb01nd an adversarial example for a point x\u00fa in our training\nset, it will indeed satisfy that sgn(g(x\u00fa + z)) \u201d= sgn(c\u00fa(x)) and correctly penalize g for the\nmistake. More generally, we could also de\ufb01ne an adversarial example as one where given pair\n(x\u00fa, y) the goal is to \ufb01nd a z such that sgn(g(x\u00fa+z)) \u201d= y. All of our guarantees from Section 3\napply to this de\ufb01nition as well. Finally, in the non-realizable case, the distinction between\nde\ufb01ning adversarial robustness as either sgn(g(x\u00fa + z)) \u201d= sgn(g(x\u00fa)), or sgn(g(x\u00fa + z)) \u201d= y,\nor even sgn(g(x\u00fa + z)) \u201d= sgn(c\u00fa(x)) matters and has dierent computational and statistical\nimplications [11, 18]. Understanding when one can achieve computationally ecient robust\nlearning in the non-realizable case is an important direction for future work.\nThe de\ufb01nition of \u201c-approximately robustly learnability has the realizability assumption built\ninto it. So, when we prove that a class F is \u201c-approximately robustly learnable, we \ufb01nd an\napproximate robust learner from F under the realizability assumption on F i.e. for a set\nof points from the distribution, the algorithm guarantees to return an approximate robust\nlearner only if there exists a perfect robust learner in the class F of learners.\n3 Finding Adversarial Examples using Polynomial Optimization\nIn this section we introduce the broad class of polynomial optimization problems which are\nuseful in designing algorithms with provable guarantees for generating adversarial examples\nfor large classes like PTFs, and will later be useful for two layer neural networks in Section 6.\nThese polynomial optimization problems are generalizations of well-studied combinatorial\noptimization problems like the Groth\u00ebndieck problem and computing operator norms of\nmatrices [19, 2, 9]. We then design algorithms with provable guarantees for some of these\nclasses. The following simple proposition illustrates the connection.\nProposition 3.1. Let \u201c \u00d8 1. There is an ecient algorithm that given a classi\ufb01er sgn(f(x))\n\u0152(x\u00fa,\u201c\u201d ), or (b)\nand a point x\u00fa, guarantees to either (a) \ufb01nd an adversarial example in Bn\n\u0152(x\u00fa,\u201d ), given an ecient algorithm\ncertify the absence of any adversarial example in Bn\nthat given x and a polynomial g(z) \u0153{ f(x\u00fa + z),\u2260f(x\u00fa + z)} \ufb01nds a \u201az such that g(\u201az) \u00d8\nmax\u00cez\u00ce\u0152\u00c6\u201d g(z) with \u00ce\u201az\u00ce\u0152 \u00c6 \u201c\u201d.\n\nWhen the classi\ufb01er is a degree-d PTF of the form sgn(f), we get the following problem:\ngiven as input a degree d polynomial g (potentially dierent from f), and any \u00f7, \u201d > 0, \ufb01nd\nin time poly(n, log( 1\n\n\u00f7)) and with probability at least 1 \u2260 \u00f7, outputs a point \u02c6x s.t.\n\ng(\u02c6x) \u00d8 max\nx\u0153Bn\n\u0152\n\ng(x) and\u201ax \u0153 Bn\nThis is closely related to the standard approximation variant of polynomial maximization\nproblem where the goal is to obtain, in polynomial time, an objective value as close to the\noptimal one, without violating the Bn\n\u0152 ball constraint. Instead, our problem asks for the\nsame objective value at the cost of an increase in the radius of the optimization ball (this is\n\n\u0152(0,\u201c\u201d ).\n\n(1)\n\n(0,\u201d)\n\n4\n\n\f1. Given (A, b, c) that de\ufb01nes the polynomial g(z) := zT Az + bT z + c.\n2. Solve the SDP given by following vector program:\n\n2 \u00c6 \u201d2 \u2019i \u0153 [n], \u00ceu0\u00ce2\n\nmax qi,j Aij\u00c8ui, uj\u00cd +qi bi\u00c8ui, u0\u00cd + c subject to \u00ceui\u00ce2\n2 = 1.\nrepresent the component of ui orthogonal to u0. Draw \u2019 \u2265 N(0, I) a\nstandard Gaussian vector, and set\u201azi := \u00c8ui, u0\u00cd+\u00c8u\u2039i ,\u2019 \u00cd for each i \u0153{ 0, 1, . . . , n}.\n\n3. Let u\u2039i\n4. Repeat rounding O(log(1/\u00f7)) random choices of \u2019 and pick the best choice.\nFigure 1: The SDP-based algorithm for the degree-2 optimization problem.\n\nsometimes called a (1,\u201c )-bicriteria approximation). This changes the \ufb02avor of the problem,\nand introduces new challenges particularly when the polynomial g is non-homogenous. We\nbegin with the following simple claim.\nClaim 3.2. There is a deterministic linear-time algorithm that given any linear threshold\nfunction sgn(bT x + c), a point x\u00fa and \u201d> 0, provably \ufb01nds an adversarial example \u00b8\u0152 ball\nof \u201d around x\u00fa when it exists.\nIn Section 4, this will be used to give robust learning algorithms for linear classi\ufb01ers. Our\nmain result of this section is a provable algorithm for degree-2 PTFs.\nTheorem 3.3. For any \u201d, \u00f7 > 0, there is a polynomial time algorithm that given a degree-2\nPTF sgn(f(x)) and a example (x\u00fa, sgn(f(x\u00fa))), guarantees at least one of the following holds\n\nwith probability at least (1 \u2260 \u00f7): (a) \ufb01nds an adversarial example (x\u00fa +\u201az) i.e., sgn(f(x\u00fa)) \u201d=\nsgn(f(x\u00fa +\u201az)), with \u00ce\u201az\u00ce\u0152 \u00c6 C\u201d\u00d4log n, or (b) certi\ufb01es that \u2019z : \u00cez\u00ce\u0152 \u00c6 \u201d, sgn(f(x\u00fa)) =\nsgn(f(x\u00fa + z)) for some constant C > 0.\nTo establish the above theorem using Proposition 3.1, we need to design a polynomial time\nalgorithm that given any degree-2 polynomial g(x) = xT Ax + bT x + c with A \u0153 Rn\u25can, b \u0153\nRn, c \u0153 R, \ufb01nds a solution\u201ax with \u00ce\u201ax\u00ce\u0152 \u00c6 O(\u00d4log n) \u00b7 \u201d such that g(\u201ax) \u00d8 max\u00cex\u00ce\u0152\u00c6\u201d g(x).\nWe design such an algorithm via an semi-de\ufb01nite programming (SDP) based approach that\nis directly inspired by the algorithm for quadratic programming (QP) by [27, 9]. However,\nfurther complications arise due to non-homogeneity, and as our goal is to preserve the\nobjective function while potentially relaxing the constraint. We defer to the appendix for\na detailed discussion. In Fig. 1 we describe the SDP that we use and the corresponding\nrounding algorithm to solve the optimization problem. The vector program given in step\n2 of Algorithm 1 is an SDP where the variables are Xij = \u00c8ui, uj\u00cd, and can be solved in\npolynomial time up to any additive error (using the Ellipsoid algorithm). We defer the the\ndetails Appendix D.\n\n4 From Adversarial Examples to Robust Learning Algorithms\nIn this section we show how to leverage algorithms for \ufb01nding adversarial examples to design\npolynomial time robust learning algorithms for degree-1 and degree-2 PTFs. We obtain our\nupper bounds by establishing a general algorithmic framework that relates robust learnability\nof PTFs to the polynomial maximization problem studied in Section 3.\nDe\ufb01nition 4.1 (\u201c-factor admissibility). For \u201c \u00d8 1, we say that a class F of PTFs is \u201c-factor\nadmissible if F has the following properties:\n(1) For any a, b, c \u0153 R s.t. sgn(f(x)), sgn(g(x)) \u0153F , we have sgn(af(x) + bg(x) + c) \u0153F .\nFurther for any r \u0153 Rn, we have sgn(g(x + r)) \u0153F .\n(2) There is a polynomial time algorithm that solves the optimization problem of maximizing\ng(x + z) around any point x, i.e., given a g \u0153F , an x and \u201d> 0, the algorithm outputs a \u02c6z\nsuch that g(x + \u02c6z) \u00d8 maxz\u0153B\u0152(0,\u201d) g(x + z)and \u00cez\u00ce\u0152 \u00c6 \u201c\u201d.\nThe \ufb01rst two conditions above are natural and are satis\ufb01ed by many classes of PTFs. The\nthird condition in the above de\ufb01nition concerns the optimization problem studied in Section 3.\nThe main result of this section, stated below, is the claim that any admissible class of PTFs\nis also robustly learnable in polynomial time.\nTheorem 4.2. Let F be a class of PTFs that is \u201c-factor admissible for \u201c \u00d8 1. Then F is\n\u201c-approximate robustly learnable.\n\n5\n\n\f1. Let S = (x1, y1), (x2, y2), . . . , (xm, ym) be the given training set.\n2. Find a degree-d polynomial g with sgn(g(x)) \u0153F that satis\ufb01es:\n\nsup\nz\u0153Bn\n\u0152\n\n(0,\u201d)\n\n\u2019i \u0153 [m],\n\n(\u2260yi)g(xi + z) < 0.\n\nd\n\n\".\n\nFigure 2: Convex program to \ufb01nd a PTF sgn(g(x)) \u0153F with zero robust empirical error.\nTo learn a g \u0153F we formulate robust empirical risk minimization as a convex program,\nshown in Figure 2. Here we use the fact that the value of any polynomial g of degree d at a\ngiven point x can be expressed as the inner product between the co-ecient vector of g and\nan appropriate vector \u00c2(x) \u0153 RD where D =!n+d\u22601\nIt is easy to see that the constraints\nin the program above are linear in the coecients of g. Furthermore, checking the validity of\neach constraint is really asking to check the robustness of g at a given point (xi, yi), which is\nan NP-hard problem [9]. Instead, we will use the fact that F is \u201c-factor admissible to design\nan approximate separation oracle for the type of constraints enforced in the program. Below\nwe give a proof sketch of Theorem 4.2 and defer the full proof to Appendix E.\nProof Sketch of Theorem 4.2. Let B be an algorithm that achieves the \u201c-factor admissibility\nfor the class F. Given S, we will run the Ellipsoid algorithm on the convex program in\nFigure 2. In each iteration, for each i \u0153 [m], we run B on the polynomial yig(xi + z), where z\nis the variable and xi is \ufb01xed to be the ith data point. From the guarantee of B we get that\nif there exists an i and z with \u00cez\u00ce\u0152 \u00c6 \u201d/\u201c, such that (\u2260yi)g(xi + z) > 0, then with high\nprobability, B will output a violated constraint of the convex program, i.e., an index i \u0153 [m]\nand \u02c6z \u0153 Bn\n\u0152(0,\u201d ) such that (\u2260yi)g(xi + \u02c6z) > 0. This gives us a separating hyperplane of the\nform sgn(\u2260yig(xi + \u02c6z)), and the algorithm continues. This means that when the algorithm\nterminates, we would have the empirical robust error \u02c6err\u201d/\u201c,S (sgn(g)) = 0. Using the uniform\nconvergence bound from Lemma A.1, this would imply that err\u201d/\u201c,D(sgn(g)) \u00c6 \u00c1.\nAs a result we get the following corollaries about linear classi\ufb01er and degree-2 PTFs. The\nproof for linear classi\ufb01ers just follows from Claim 3.2, and Theorem 3.3 immediately implies\nthe result for degree-2 PTFs.\nCorollary 4.3. The class of linear classi\ufb01ers is optimally robustly learnable. The class of\ndegree-2 PTFs is O(\u00d4log n)-approximately robustly learnable.\n\n5 Computational Intractability of Learning Robust Classi\ufb01ers\nIn this section, we leverage the connection to polynomial optimization to complement\nour upper bound with the following nearly matching lower bound.We give a reduction\nfrom Quadratic Programming (QP) where given a polynomial p(x) = qi<j aijxixj, and\na value s, the goal is to distinguish whether maxx\u0153{\u22601,1}np(x) < s or whether exists an\nx such that p(x) > s\u00f7 approx.\nIt is known that the distinguishing problem is hard for\n\u00f7approx = O(logc n) for some constant c > 0 [3]; moreover the state-of-the-art algorithms\ngive a \u00f7approx = O(log n) factor approximation [9] and improving upon this factor is a major\nopen problem. By appropriately scaling the instance, this immediately implies the hardness\nof checking whether a given degree-2 PTF is robust around a given point.\nHowever, this does not suce for hardness of learning, since given a distribution supported\nat a single point, there is a trivial constant classi\ufb01er that robustly classi\ufb01es the instance\ncorrectly. More generally, there could exist a dierent degree-2 PTF that could be easy to\ncertify for the given point.\nInstead, given a degree-2 PTF sgn(p(x)), we carefully construct\na set of O(n2) points such that any classi\ufb01er that is robust on an instance supported on\nthe set will have to be close to the given polynomial p. Having established this, we can\ndistinguish between the two cases of the QP problem by whether the learning algorithm is\nable to output a robust classi\ufb01er or not. This is formalized below.\nTheorem 5.1. There exists \u201d, \u00c1 > 0, such that assuming N P \u201d= RP there is no algorithm\nthat given a set of N = poly(n, 1\n\u00c1) samples from a distribution D over Rn \u25ca {\u22601, +1}, runs\nin time poly(N) and distinguishes between the following two cases for any \u201d\u00d5 = o(\u00d4\u00f7approx\u201d):\n\n\u2022 Yes: There exists a degree-2 PTF that has \u201d-robust error of 0 w.r.t. D.\n\n6\n\n\f\u2022 No: There exists no degree-2 PTF that has \u201d\u00d5-robust error at most \u00c1 w.r.t. D.\n\nHere \u00f7approx is the hardness of approximation factor of the QP problem.\nRemark 5.2. The above theorem proves that any polynomial time algorithm that always\noutputs a robust classi\ufb01er (or declares failure if it does not \ufb01nd one) will have to incur an\nextra factor of (\u00d4\u00f7approx) in the robustness parameter \u201d. Our upper bound in Section 4\non the other hand matches this bound. While our lower bound applies to algorithms that\noutput a classi\ufb01er of low error, in Appendix (see Theorem G.6) we also prove a more robust\nlower bound that rules out the possibility of an ecient robust learner that incurs an error\nless than 1/4.\n\n6 Finding Adversarial Examples for Two Layer Neural Networks\nNext we use the framework in Section 3 to design new algorithms for \ufb01nding adversarial\nexamples in two layer neural networks with ReLU activations. We describe the setting\nfor binary classi\ufb01cation below. A two layer neural network with ReLU gates is given\nby parameters (v1, v2, W) and outputs f1(x) = vT1 \u2021(W x), f2(x) = vT2 \u2021(W x) where x \u0153\nRn, v1, v2 \u0153 Rk and W \u0153 Rk\u25can. Here \u2021 : Rm \u00e6 Rm is a co-ordinate wise non-linear operator\n\u2021(yi) = max { 0, yi } for each i \u0153 [m]. The binary classi\ufb01er corresponding to the network\nis sgn(f1(x) \u2260 f2(x)) = sgn(vT \u2021(W x)) where v = v1 \u2260 v2. The optimization problem that\narises is the following: given an instance with A \u0153 Rm1\u25can,\u2014 \u0153 Rm2, B \u0153 Rm2\u25can, c1 \u0153\nRn, c2 \u0153 Rm1, c0 \u0153 R, the goal is to \ufb01nd opt(A, B, \u2014, c), de\ufb01ned as :\n\nopt(A, B, \u2014, c) := max\n\nz:\u00cez\u00ce\u0152\u00c6\u201d\u00cec2 + Az\u00ce1 + cT\n\n1 z \u2260 \u00ce\u2014 + Bz\u00ce1 + c0\n\n= max\n\nz:\u00cez\u00ce\u0152\u00c6\u201d\n\nmax\n\ny:\u00cey\u00ce\u0152\u00c61\n\nyT Az + cT\n\n1 z + cT\n\n2 y \u2260\n\nm2\u00ffj=1|\u2014j + BT\n\nj z|.\n\n(2)\n\n\u0152(x\u00fa,\u201d ).\n\nHere Bj is the jth row of B. Let c denote (c0, c1, c2), and let opt(A, B, \u2014, c) be the optimal\nvalue of the above problem. The following proposition holds in a slightly more general setting\nwhere there can be an extra linear term as described below.\nProposition 6.1. Let \u201c \u00d8 1. Suppose there is an ecient algorithm that given an instance\nof problem (2) \ufb01nds a solution \u201az,\u201ay with \u00ce\u201az\u00ce\u0152 \u00c6 \u201c\u201d, \u00ce\u201ay\u00ce\u0152 \u00c6 1 such that f(\u201ay,\u201az) > 0 when\nopt(A, b, \u2014, c) > 0. Then there is an ecient algorithm that given a two layer neural net\nsgn(f(x)) where f(x) := vT \u2021(W x) + (v\u00d5)T x and an example x\u00fa, guarantees to either (a)\n\u0152(x\u00fa,\u201c\u201d ) around x\u00fa, or (b) certify the absence of any\n\ufb01nd an adversarial example in the Bn\nadversarial example in Bn\nOur algorithm for solving (2) given in Figure 3 is inspired by Algorithm 1 for polynomial\noptimization. However, the rounding algorithm diers because the variables yj and variables\nzi serve dierent purposes in (2), and we need to simultaneously satisfy dierent constraints\non them to produce a valid perturbation. Moreover when the SDP is negative, then this\ngives a certi\ufb01cate of robustness around x.\nPlease see Section F for a simple proof and more details. We remark that one can obtain\nprovable guarantees similar to Theorem 4.2 for Algorithm 3 under certain regularity conditions\nabout the SDP solution. However, this is unsatisfactory as this depends on the SDP solution\nto the given instance, as opposed to an explicit structural property of the instance. Obtaining\nprovable guarantees of the latter kind is an interesting open question.\nExperiments\nNext, we evaluate the performance of the proposed attack in Figure 3 and compare it with\nthe state of the art projected gradient descent(PGD) based attack of Madry et al. [24]. Our\napproach indeed \ufb01nds more adversarial examples, although at a higher computational cost\nsince we need to solve an SDP per example and per pair of classes. We use the MNIST data\nset and our 2-layer neural network has d = 784 input units, k = 1024 hidden units and 10\noutput units. The SDP has d + k + 1 vector variables, and takes about 200s per instance on\na standard desktop. Hence we perform our experiments on randomly chosen subsets of the\nMNIST data set. Another optimization we perform for computational reasons is that given\n\n7\n\n\frj + c0\n\nn\u00ffi=1\n\nAj,i\u00c8vj, ui\u00cd +\n\n1. Given instance I = (A, B, \u2014, c) of (2), solve SDP with parameter \u00f7 \u0153 (0, 1):\nm1\u00ffj=1\nsdp = max \u00ffj\u0153[m1],i\u0153[n]\nc2(j)\u00c8u0, vj\u00cd \u2260 \u00ffj\u0153[m2]\nc1(i)\u00c8ui, u0\u00cd +\nand \u00ceu0\u00ce2 = 1\ns.t.\u2019j \u0153 [m1] \u00cevi\u00ce2 \u00c6 1, \u2019i \u0153{ 1, . . . , n}\u00ce ui\u00ce2 \u00c6 \u201d2,\nBj,i\u00c8ui, u0\u00cd).\n\u2019j \u0153 [k2]\n\nrj \u00d8 (\u2014j +\u00ffj\n2. Let u\u2039i , v\u2039j represent the components of ui, vj orthogonal to u0. Let \u00c1 \u0153 (0, 1)\nwith \u00c1 = (1)/\u00d4log m1. Let \u2019 \u2265 N(0, I) be a Gaussian vector; set \u2019i \u0153\n{ 0, 1, . . . , n} , \u201azi := \u00c8ui, u0\u00cd + 1\n3. Repeat rounding with poly(n) random choices of \u2019 and pick the best choice.\n\nBj,i\u00c8ui, u0\u00cd), and rj \u00d8 \u2260(\u2014j +\u00ffj\n\u00c1\u00c8u\u2039i ,\u2019 \u00cd, \u201ayj := \u00c8vj, u0\u00cd + \u00c1\u00c8v\u2039j ,\u2019 \u00cd.\n\nFigure 3: The SDP-based algorithm for Problem (2).\nPGDpass (6 \u25ca 50 random samples) PGDfail (8 \u25ca 100 random samples)\nMean : 49.5 out of 50, Std : 0.76\nMean 30.6 out of 100, Std : 2.87\n\n297 out of 300 total\n\n244 out of 800 total\n\n\u201d = 0.3\n\nSDP succeeds\n\n\u201d = 0.01\n\nPGDpass (138 samples)\n\nPGDfail (100 ranked)\n\nSDP succeeds\nTable 1: For \u201d = 0.3, we report mean and standard deviation across batches of the number of\nadversarial examples found by running our SDPattack algorithm on 6 batches of 50 random examples\nfrom PGDpass and 8 batches of 100 random samples from PGDfail. For \u201d = 0.01, we run SDPattack\non all 138 examples in PGDpass and \ufb01rst 100 sorted examples from PGDfail.\n\n138\n\n45\n\nan example x with predicted class i, we use a greedy heuristic to pick a class j \u201d= i for the\npotential adversarial example x+ z. So the numbers we report below are an underestimate of\nthe eectiveness of the full SDP based algorithm. See Appendix B for a detailed discussion.\nWe consider two settings of the parameter \u201d, the maximum amount by which each pixel can\nbe perturbed to produce a valid attack example. As in [24] we \ufb01rst choose \u201d = 0.3 and train a\nrobust 2-layer network using the algorithm in [24]. This network has an accuracy of 82.32%\nand adversarial accuracy (allowing for adversarial perturbations) of 31.7% on the test set.\nWe then run the PGD attack and divide the test set into examples where the PGD attack\nsucceeds (PGDPass) and examples where the PGD attack fails (PGDfail). We then run our\nattack on batches of random subsets chosen from each set. The \ufb01rst row of Table 1 shows the\nprecision and recall of our method, along with the average and the standard deviation across\nthe chosen batches. As one can see, our method has very high recall, i.e., whenever the PGD\nattack succeeds, our SDP based algorithm also \ufb01nds adversarial examples. Furthermore, on\nexamples where the PGD attack fails, our method is still able to discover new adversarial\nexamples 30% of the time. See a sample of the perturbed images produced by our method\nin Section B. In particular, Figure 4 shows images of some of the examples where the SDP\nbased attack succeeds, but the PGDattack fails and Figure 5 shows some images where\nboth the PGDattack and SDP based attack succeed. A visual inspection of both the \ufb01gures\nreveals that our attack often produces sparse targeted attacks as opposed to PGDattack.\nWe also run the PGD attack on the network with \u201d = 0.01. Here we notice that attack\nsucceeds on only 138 test examples and hence we can aord to run our attack on all of\nthem. As can be seen from the second row of Table 1 our attack succeeds on all of these\nexamples. Further, when we run our algorithm on the \ufb01rst 100 examples from PGDfail\npicked according to a greedy heuristic (see Section B for details), our method \ufb01nds 45 new\nadversarial examples. This implies at least a (138 + 45)/138 = 1.33-fold advantage here.\nThe experiments above suggest that our theoretical claims and algorithms can lead to\nimproved attacks. We would like to note that the recent work of [29] also studied SDP based\nmethods for providing adversarial certi\ufb01cates for 2-layer neural networks. However, our SDP\nas outlined in Figure 3 is strictly stronger. The SDP of [29] is in fact independent of the\ngiven example x, so we expect our method to produce better certi\ufb01cates. We leave as future\nwork the task of making our theoretical analysis practical for large scale applications.\n\n8\n\n\f7 Future Directions\nDesign of polynomial time algorithms that provably achieve adversarial robustness is an\nimportant direction of research. Several open questions remain to be explored further. In\nSection 4 we provide a general algorithmic framework for designing polynomial time robust\nalgorithms. It would be interesting to use our framework to design robust algorithms for\ngeneral degree-d PTFs. While there are algorithms to approximately maximize degree-d\npolynomials, they focus on the homogeneous case which does not suce for our purposes.\nAnother important direction for future work is to convert our adversarial attack algorithm\nfor 2-layer neural networks into a provably robust learning algorithm via the framework\nof Section 4. A straightforward invocation of the framework does not lead to a convex\nconstraint set. It would also be interesting to design provable adversarial attacks for higher\ndepth networks. Finally, our experimental results suggest that making our SDP based attack\nwork on a large scale could lead to improved adversarial attacks.\n\nAcknowledgements\nThe second and third authors were supported by the National Science Foundation (NSF)\nunder Grant No. CCF-1652491 and CCF-1637585. Additionally, the second author was\nfunded by the Morrison Fellowship from Northwestern University.\n\nReferences\n[1] Noga Alon, Konstantin Makarychev, Yury Makarychev, and Assaf Naor. Quadratic\n\nforms on graphs. Inventiones mathematicae, 163(3):499\u2013522, 2006.\n\n[2] Noga Alon and Assaf Naor. Approximating the cut-norm via grothendieck\u2019s inequality.\nIn Proceedings of the thirty-sixth annual ACM symposium on Theory of computing,\npages 72\u201380. ACM, 2004.\n\n[3] Sanjeev Arora, Eli Berger, Hazan Elad, Guy Kindler, and Muli Safra. On non-\napproximability for quadratic programs. In Foundations of Computer Science, 2005.\nFOCS 2005. 46th Annual IEEE Symposium on, pages 206\u2013215. IEEE, 2005.\n\n[4] Idan Attias, Aryeh Kontorovich, and Yishay Mansour. Improved generalization bounds\n\nfor robust learning. arXiv preprint arXiv:1810.02180, 2018.\n\n[5] Chiranjib Bhattacharyya. Robust classi\ufb01cation of noisy data using second order cone\nIn Intelligent Sensing and Information Processing, 2004.\n\nprogramming approach.\nProceedings of International Conference on, pages 433\u2013438. IEEE, 2004.\n\n[6] Alberto Bietti, Gr\u00e9goire Mialon, and Julien Mairal. On regularization and robustness\n\nof deep neural networks. arXiv preprint arXiv:1810.00363, 2018.\n\n[7] S\u00e9bastien Bubeck, Yin Tat Lee, Eric Price, and Ilya Razenshteyn. Adversarial examples\nfrom cryptographic pseudo-random generators. arXiv preprint arXiv:1811.06418, 2018.\n[8] S\u00e9bastien Bubeck, Eric Price, and Ilya Razenshteyn. Adversarial examples from compu-\n\ntational constraints. arXiv preprint arXiv:1805.10204, 2018.\n\n[9] M Charikar and A Wirth. Maximizing quadratic programs: extending grothendieck\u2019s\ninequality. In Foundations of Computer Science, 2004. Proceedings. 45th Annual IEEE\nSymposium on, pages 54\u201360. IEEE, 2004.\n\n[10] Daniel Cullina, Arjun Nitin Bhagoji, and Prateek Mittal. Pac-learning in the presence\n\nof evasion adversaries. arXiv preprint arXiv:1806.01471, 2018.\n\n[11] Dimitrios Diochnos, Saeed Mahloujifar, and Mohammad Mahmoody. Adversarial risk\nand robustness: General de\ufb01nitions and implications for the uniform distribution. In\nAdvances in Neural Information Processing Systems, pages 10380\u201310389, 2018.\n\n9\n\n\f[12] Alhussein Fawzi, Seyed-Mohsen Moosavi-Dezfooli, and Pascal Frossard. Robustness\nof classi\ufb01ers: from adversarial to random noise. In Advances in Neural Information\nProcessing Systems, pages 1632\u20131640, 2016.\n\n[13] Uriel Feige, Yishay Mansour, and Robert Schapire. Learning and inference in the\npresence of corrupted inputs. In Conference on Learning Theory, pages 637\u2013657, 2015.\n[14] Michael R Garey and David S Johnson. Computers and intractability, volume 29. wh\n\nfreeman New York, 2002.\n\n[15] Justin Gilmer, Ryan P Adams, Ian Goodfellow, David Andersen, and George E Dahl.\nMotivating the rules of the game for adversarial example research. arXiv preprint\narXiv:1807.06732, 2018.\n\n[16] Justin Gilmer, Luke Metz, Fartash Faghri, Samuel S Schoenholz, Maithra Raghu, Martin\nWattenberg, and Ian Goodfellow. Adversarial spheres. arXiv preprint arXiv:1801.02774,\n2018.\n\n[17] Amir Globerson and Sam Roweis. Nightmare at test time: robust learning by feature\ndeletion. In Proceedings of the 23rd international conference on Machine learning, pages\n353\u2013360. ACM, 2006.\n\n[18] Pascale Gourdeau, Varun Kanade, Marta Kwiatkowska, and James Worrell. On the\n\nhardness of robust classi\ufb01cation. arXiv preprint arXiv:1909.05822, 2019.\n\n[19] A. Grothendieck and V. Losert. \"R\u00e9sum\u00e9 de la th\u00e9orie m\u00e9trique des produits tensoriels\n\ntopologiques\". Univ., 1976.\n\n[20] Michael J Kearns, Umesh Virkumar Vazirani, and Umesh Vazirani. An introduction to\n\ncomputational learning theory. MIT press, 1994.\n\n[21] Justin Khim and Po-Ling Loh. Adversarial risk bounds for binary classi\ufb01cation via\n\nfunction transformation. arXiv preprint arXiv:1810.09519, 2018.\n\n[22] Subhash Khot and Assaf Naor. Linear equations modulo 2 and the l1 diameter of\nconvex bodies. In Foundations of Computer Science, 2007. FOCS\u201907. 48th Annual\nIEEE Symposium on, pages 318\u2013328. IEEE, 2007.\n\n[23] Subhash Khot and Ryan O\u2019Donnell. Sdp gaps and ugc-hardness for maxcutgain. In\nFoundations of Computer Science, 2006. FOCS\u201906. 47th Annual IEEE Symposium on,\npages 217\u2013226. IEEE, 2006.\n\n[24] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian\nVladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint\narXiv:1706.06083, 2017.\n\n[25] Saeed Mahloujifar, Dimitrios I Diochnos, and Mohammad Mahmoody. The curse of\nconcentration in robust learning: Evasion and poisoning attacks from concentration of\nmeasure. arXiv preprint arXiv:1809.03063, 2018.\n\n[26] Saeed Mahloujifar and Mohammad Mahmoody. Can adversarially robust learning\n\nleverage computational hardness? arXiv preprint arXiv:1810.01407, 2018.\n\n[27] Yu Nesterov. Semide\ufb01nite relaxation and nonconvex quadratic optimization. Optimiza-\n\ntion methods and software, 9(1-3):141\u2013160, 1998.\n\n[28] Ryan O\u2019Donnell. Analysis of Boolean Functions. Cambridge University Press, New\n\nYork, NY, USA, 2014.\n\n[29] Aditi Raghunathan, Jacob Steinhardt, and Percy Liang. Certi\ufb01ed defenses against\n\nadversarial examples. arXiv preprint arXiv:1801.09344, 2018.\n\n[30] Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, and Aleksander\narXiv preprint\n\nMadry. Adversarially robust generalization requires more data.\narXiv:1804.11285, 2018.\n\n10\n\n\f[31] Pannagadatta K Shivaswamy, Chiranjib Bhattacharyya, and Alexander J Smola. Second\norder cone programming approaches for handling missing and uncertain data. Journal\nof Machine Learning Research, 7(Jul):1283\u20131314, 2006.\n\n[32] Aman Sinha, Hongseok Namkoong, and John Duchi. Certifying some distributional\n\nrobustness with principled adversarial training. 2018.\n\n[33] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian\nGoodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint\narXiv:1312.6199, 2013.\n\n[34] Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander\n\nMadry. Robustness may be at odds with accuracy. 2018.\n\n[35] Eric Wong and Zico Kolter. Provable defenses against adversarial examples via the\nconvex outer adversarial polytope. In International Conference on Machine Learning,\npages 5283\u20135292, 2018.\n\n[36] Huan Xu, Constantine Caramanis, and Shie Mannor. Robustness and regularization of\nsupport vector machines. Journal of Machine Learning Research, 10(Jul):1485\u20131510,\n2009.\n\n[37] Huan Xu and Shie Mannor. Robustness and generalization. Machine learning, 86(3):391\u2013\n\n423, 2012.\n\n[38] Dong Yin, Kannan Ramchandran, and Peter Bartlett. Rademacher complexity for\n\nadversarially robust generalization. arXiv preprint arXiv:1810.11914, 2018.\n\n11\n\n\f", "award": [], "sourceid": 7666, "authors": [{"given_name": "Pranjal", "family_name": "Awasthi", "institution": "Rutgers University/Google"}, {"given_name": "Abhratanu", "family_name": "Dutta", "institution": "Northwestern University"}, {"given_name": "Aravindan", "family_name": "Vijayaraghavan", "institution": "Northwestern University"}]}