{"title": "Nonzero-sum Adversarial Hypothesis Testing Games", "book": "Advances in Neural Information Processing Systems", "page_first": 7312, "page_last": 7322, "abstract": "We study nonzero-sum hypothesis testing games that arise in the context of adversarial classification, in both the Bayesian as well as the Neyman-Pearson frameworks. We first show that these games admit mixed strategy Nash equilibria, and then we examine some interesting concentration phenomena of these equilibria. Our main results are on the exponential rates of convergence of classification errors at equilibrium, which are analogous to the well-known Chernoff-Stein lemma and Chernoff information that describe the error exponents in the classical binary hypothesis testing problem, but with parameters derived from the adversarial model. The results are validated through numerical experiments.", "full_text": "Nonzero-sum Adversarial Hypothesis Testing Games\n\nDepartment of Electrical Communication Engineering\n\nSarath Yasodharan\n\nIndian Institute of Science\nBangalore 560 012, India\n\nsarath@iisc.ac.in\n\nUniv. Grenoble Alpes, Inria, CNRS, Grenoble INP, LIG & MPI-SWS\n\nPatrick Loiseau\n\n700 avenue Centrale\nDomaine Universitaire\n\n38400 St Martin d\u2019H\u00e9res, France\npatrick.loiseau@inria.fr\n\nAbstract\n\nWe study nonzero-sum hypothesis testing games that arise in the context of adversar-\nial classi\ufb01cation, in both the Bayesian as well as the Neyman-Pearson frameworks.\nWe \ufb01rst show that these games admit mixed strategy Nash equilibria, and then\nwe examine some interesting concentration phenomena of these equilibria. Our\nmain results are on the exponential rates of convergence of classi\ufb01cation errors\nat equilibrium, which are analogous to the well-known Chernoff-Stein lemma\nand Chernoff information that describe the error exponents in the classical binary\nhypothesis testing problem, but with parameters derived from the adversarial model.\nThe results are validated through numerical experiments.\n\n1\n\nIntroduction\n\nClassi\ufb01cation is a simple but important task that has numerous applications in a variety of domains\nsuch as computer vision or security. A traditional assumption that is used in the design of classi\ufb01cation\nalgorithms is that the input data is generated without knowledge of the classi\ufb01er being used, hence\nthe data distribution is independent of the classi\ufb01cation algorithm. This assumption is no longer valid\nin the presence of an adversary, as an adversarial agent can learn the classi\ufb01er and deliberately alter\nthe data such that the classi\ufb01er makes an error. This is the case in particular in security applications\nwhere the classi\ufb01er\u2019s goal is to detect the presence of an adversary from the data it observes.\nAdversarial classi\ufb01cation has been studied in two main settings. The \ufb01rst focuses on adversarial\nversions of a standard classi\ufb01cation task in machine learning, where the adversary attacks the\nclassi\ufb01er (defender/decision maker) by directly choosing vectors from a given set of data vectors;\nwhereas the second focuses on adversarial hypothesis testing, where the adversary (attacker) gets to\nchoose a distribution from a set of distributions and independent data samples are generated from\nthis distribution. The main differences of the latter framework from the former are that: (i) the\nadversary only gets to choose a distribution (rather than the actual attack vector) and data is generated\nindependently from this distribution, and (ii) the defender makes a decision only once after it observes\na whole data sequence instead of making a decision for each individual data sample it receives. Both\nof these frameworks have applications in a variety of domains, but prior literature has mainly focused\non the \ufb01rst setting; see Section 1.1 for a description of the related literature.\n\n33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada.\n\n\fIn this paper, we focus on the setting of adversarial hypothesis testing. To model the interaction\nbetween the attacker and defender, we formulate a nonzero-sum two-player game between the\nadversary and the classi\ufb01er where the adversary picks a distribution from a given set of distributions,\nand data is generated independently from that distribution (a non-attacker always generates data from\na \ufb01xed distribution). The defender on his side makes a decision based on observation of n data points.\nOur model can also be viewed as a game-theoretic extension of the classical binary hypothesis testing\nproblem where the distribution under the alternate hypothesis is chosen by an adversary. Based on\nour game model, we are then able to extend to the adversarial setting the main results of the classical\nhypothesis testing problem (see Section 2) on the form of the best decision rule and on the rates of\ndecrease of classi\ufb01cation errors. More speci\ufb01cally, our contributions can be summarized as follows:\n1. We propose nonzero-sum games to model adversarial hypothesis testing problems in a\n\n\ufb02exible manner.\n\n2. We show existence of mixed strategy Nash equilibria in which the defender employs certain\nlikelihood ratio tests similar to that used in the classical binary hypothesis testing problem.\n3. We show that the classi\ufb01cation errors under all Nash equilibria for our hypothesis testing\ngames decay exponentially fast in the number of data samples. We analytically obtain these\nerror exponents, and it turns out that they are same as those arising in certain classical\nhypothesis testing problem, with parameters derived from the adversarial model.\n\n4. We illustrate the results, in particular the importance of some assumptions, using simulations.\nThroughout our analysis, an important dif\ufb01culty lies in that the strategy spaces of both the players are\nuncountable; we believe, however, that it is an important feature of the model to be realistic.\n\n1.1 Related Work\n\nAdversarial classi\ufb01cation and the security of machine learning have been studied extensively in the\npast decade, see e.g., [9, 20, 4, 14, 18, 22]; here we focus only on game-theoretic approaches to tackle\nthe problem. Note that, besides the adversarial learning problem, game theory has been successfully\nused to tackle several other security problems such as allocation of monitoring resources to protect\ntargets, see e.g., [8, 17]. We review here only papers relating to classi\ufb01cation.\nA number of game-theoretic models have appeared in the past decade to study the adversarial\nclassi\ufb01cation problem in the classical setting of classi\ufb01cation tasks in machine learning. [9] studies\nthe best response in an adversarial classi\ufb01cation game, where the adversary is allowed to alter training\ndata. A number of zero-sum game models were also proposed where the attacker is restricted on the\namount of modi\ufb01cations he can do to the training set, see [16, 29, 28]. [6] studies the problem of\nchoosing the best linear classi\ufb01er in the presence of an adversary (a similar model is also studied in\n[7]) using a nonzero-sum game, and shows the existence of a unique pure strategy Nash equilibrium.\nSimilar to our formulation, the strategy sets in this case are uncountable, and therefore showing the\nexistence and uniqueness of Nash equilibrium needs some work. However, in our formulation, there\nmay not always exist a Nash equilibrium in pure strategies, which makes the subsequent analysis of\nerror exponents more dif\ufb01cult. [19] studies an adversarial classi\ufb01cation game where the utilities of\nthe players are de\ufb01ned by using ROC curves. The authors study Nash equilibria for their model and\nprovide numerical discretization techniques to compute the equilibria. [12] studies a nonzero-sum\nadversarial classi\ufb01cation game where the defender has no restriction on the classi\ufb01er, but the attacker\nis limited to a \ufb01nite set of vectors. The authors show that the defender can, at equilibrium, use only a\nsmall subset of \u201cthreshold classi\ufb01ers\u201d and characterize the equilibrium through linear programming\ntechniques. In our model, the utility functions share similarities with that of [12], but we work in\nthe hypothesis testing framework and with uncountable action sets, which completely modi\ufb01es the\nanalysis. Several studies appeared recently on \u201cstrategic classi\ufb01cation\u201d, where the objective of the\nattacker(s) is to improve the classi\ufb01cation outcome in his own direction, see [13, 11].\nOn the other hand, adversarial hypothesis testing has been studied by far fewer authors. [2] studies a\nsource identi\ufb01cation game in the presence of an adversary, where the classi\ufb01er needs to distinguish\nbetween two source distributions P0 and P1 in which the adversary can corrupt samples from P0\nbefore it reaches the classi\ufb01er. They show that the game has an asymptotic Nash equilibrium when\nthe number of samples becomes large, and compute the error exponent associated with the false\nnegative probability. [3] and [26] study further extensions of this framework.\nA (non game-theoretic) hypothesis testing problem in an adversarial setting has been studied by [5],\nwhich is the closest to our work. Here, there are two sets of probability distributions and nature\n\n2\n\n\foutputs a \ufb01xed number of independent samples generated by using distributions from either one of\nthese two sets. The goal of the classi\ufb01er is to detect the true state of nature. The authors derive error\nexponents associated with the classi\ufb01cation error, in both Bayesian and Neyman-Pearson frameworks\nusing a worst-case maxmin analysis. Although we restrict to i.i.d. samples and let the non-attacker\nplay a single distribution, we believe that our nonzero-sum game model with \ufb02exible utilities can\nbetter capture the interaction between adversary and classi\ufb01er. There also exists extensive prior work\nwithin the statistics literature [15] on minimax hypothesis testing, which relates to our paper, but we\ndefer a discussion of how our work differs from it to after we have exposed the details of our model.\nGame-theoretic models were also used to study adversarial classi\ufb01cation in a sequential setting,\nsee [25, 1, 21], but with very different techniques and results.\n\n2 Basic Setup and Hypothesis Testing Background\n\nIn this section, we present the basic setup results in classical binary hypothesis testing.\nThroughout the paper, we consider an alphabet set X that we assume \ufb01nite. In a classical hypothesis\ntesting problem, we are given two distribution p and q, and a realization of a sequence of independent\nand identically distributed random variables X1, . . . , Xn, which are distributed as either p (under\nhypothesis H0) or q (under hypothesis H1). Our goal is to distinguish between the two alternatives:\n\nH0 : X1, X2, . . . , Xn i.i.d. \u223c p versus H1 : X1, X2, . . . , Xn i.i.d. \u223c q.\n\nIn this setting, we could make two possible types of errors: (i) we declare H1, whereas the true state\nof nature is H0 (Type I error, or false alarm), and (ii) we declare H0 whereas the true state of nature\nis H1 (Type II error, or missed detection). Note that one can make one of these errors arbitrarily\nsmall at the expense of the other by always declaring H0 or H1.\nThe trade-off between the two types of errors can be captured using two frameworks. If we have\nknowledge on the prior probabilities of the two hypotheses, then we can seek a decision rule that\nminimizes the average probability of error (this is the Bayesian framework). On the other hand, if we\ndo not have any information on the prior probabilities, then we can \ufb01x \u03b5 > 0 and seek a decision\nrule that minimizes the Type II error among all decision rules whose Type I error is at most \u03b5 (this\nis the Neyman-Pearson framework). In both of these frameworks, it can be shown that the optimal\ntest is a likelihood ratio test, i.e., given xn = (x1, . . . , xn) we compute the likelihood ratio q(xn)\np(xn)\nand compare it to a threshold to make a decision (with possible randomization at the boundary in\nthe Neyman-Pearson framework). Here, p(xn) (resp. q(xn)) denotes the probability of observing\nthe n-length word xn under the distribution p (resp. q). See Section II.B and II.D in [23] for an\nintroduction to hypothesis testing.\nFor large enough n, by the law of large numbers, the fraction of i in an observation xn is very close\nto p(i) (resp. q(i)) under H0 (resp. under H1), for each i \u2208 X . Therefore, one anticipates that the\nprobability of correct decision is very close to 1 for large enough n. Hence, one can study the rate\nat which the errors go to 0 as n becomes large. It is shown that, under both frameworks, the error\ndecays exponentially in n. In the Bayesian framework, the error exponent associated with the average\n0(\u00b7) is the Fenchel-Legendre transform of the log-moment\nprobability of error is \u2212\u039b\u2217\np(X) under H0, i.e., when X \u223c p. In the Neyman-Pearson\ngenerating function of the random variable q(X)\ncase, the error exponent associated with the Type II error is \u2212D(p||q) where D is the relative entropy\nfunctional. The above error exponents are known as Chernoff information and Chernoff-Stein lemma,\nrespectively (see Section 3.4 in [10] for the analysis on error exponents).\nIn this work, we propose extensions of the classical hypothesis testing framework to an adversarial\nscenario modeled as a game, both in the Bayesian and in the Neyman-Pearson frameworks; and we\ninvestigate how the corresponding results are modi\ufb01ed. Due to space constraints, we present only\nthe model and results for the Bayesian framework in the main body of the paper. The corresponding\nanalysis for the Neyman-Pearson framework follows similar ideas and is relegated to Appendix A of\nthe full version of this paper [27]. The proofs of all results presented in the paper (and in Appendix A\nof the the full version [27]) can be found in Appendix B of the full version [27].\n\n0(0), where \u039b\u2217\n\n3\n\n\f3 Hypothesis Testing Game in the Bayesian Framework\n\nIn this section, we formulate a one-shot adversarial hypothesis testing game in the Bayesian frame-\nwork, motivated by security problems where there might be an attacker who modi\ufb01es the data\ndistribution and a defender who tries to detect the presence of the attacker. Game theoretic modelling\nof such problems has found great success in understanding the behavior of the agents via equilibrium\nanalysis in many applications, see Section 1.1. We \ufb01rst present the model and then elaborate on its\nmotivations and on how it relates to related works in statistics.\n\n3.1 Problem Formulation\nLet X = {0, 1, . . . , d \u2212 1} denote the alphabet set with cardinality d, and let M1(X ) denote the\nspace of probability distributions on X . Fix n \u2265 1.\nThe game is played as follows. There are two players: the external agent and the defender. The\nexternal agent can either be a non-attacker or an attacker. In the Bayesian framework, we assume that\nthe external agent is an attacker with probability \u03b8, and a non-attacker (normal user) with probability\n1\u2212 \u03b8. The non-attacker is not strategic and she does not have any adversarial objective. If the external\nagent is a non-attacker, she generates n samples independently from the distribution p. If the external\nagent is an attacker, she picks a distribution q from a set of distributions Q \u2286 M1(X ) and generates\nn samples independently from q. The defender, upon observing the n-length word generated by the\nexternal agent, wants to detect the presence of the attacker.\nThroughout the paper, a decision rule implemented by the defender is denoted by \u03d5 : X n \u2192 [0, 1],\nwith the interpretation that \u03d5(xn) is the probability with which hypothesis H1 is accepted (i.e.,\nthe presence of an adversary is declared) when the defender observes the n-length word xn =\n(x1, . . . , xn). We say that a decision rule \u03d5 is deterministic if \u03d5(xn) \u2208 {0, 1} for all xn \u2208 X n.\nTo de\ufb01ne the game, let the attacker\u2019s strategy set be Q \u2286 M1(X ), and that of the defender be\n\n\u03a6n = {\u03d5 : X n \u2192 [0, 1]},\n\nwhich is the set of all randomized decision rules on n-length words.\nTo de\ufb01ne the utilities, consider the attacker \ufb01rst. We assume that there is a cost associated with\nchoosing a distribution from Q which we model using a cost function c : Q \u2192 R+. The goal of the\nattacker is to fool the defender as much as possible, i.e., he wants to maximize the probability that the\ndefender classi\ufb01es an n-length word as coming from the non-attacker whereas it is actually being\ngenerated by the attacker. To capture this, the utility of the attacker when she plays the pure strategy\nq \u2208 Q and the defender plays the pure strategy \u03d5 \u2208 \u03a6n is de\ufb01ned as\n\nuA\nn (q, \u03d5) =\n\n(1 \u2212 \u03d5(xn))q(xn) \u2212 c(q),\n\n(3.1)\n\n(cid:88)\n\nxn\n\n(cid:32)(cid:88)\n\n(cid:88)\n\n(cid:33)\n\nwhere q(xn) denotes the probability of observing the n-length word xn when the symbols are\ngenerated independently from the distribution q.\nFor the defender, the goal is to minimize the classi\ufb01cation error. Similar to the classical hypothesis\ntesting problem, there could be two types of errors: (i) the external agent is actually a non-attacker\nwhereas the defender declares that there is an attack (Type I error, or false alarm), and (ii) the external\nagent is an attacker whereas the defender declares that there is no attack (Type II error, or missed\ndetection). The goal of the defender is to minimize a weighted sum of the above two types of errors.\nAfter suitable normalization, we de\ufb01ne the utility of the defender as\n\nn (q, \u03d5) = \u2212\nuD\n\n(1 \u2212 \u03d5(xn))q(xn) + \u03b3\n\n\u03d5(xn)p(xn)\n\n,\n\n(3.2)\n\nxn\n\nxn\n\nwhere \u03b3 > 0 is a constant that captures the exogenous probability of attack (i.e., \u03b8), as well as the\nrelative weights given to the error terms.\nWe denote our Bayesian hypothesis testing game with utility functions (3.1) and (3.2) by GB(d, n).\nWith a slight abuse of notation, we denote by uA\nn ), the utility of the players\nunder a mixed strategy (\u03c3A\nFor our analysis of game GB(d, n), we will make use of the following assumptions:\n\nn ) and uD\nn \u2208 M1(Q), and \u03c3D\n\nn (\u03c3A\nn , \u03c3D\nn \u2208 M1(\u03a6n).\n\nn (\u03c3A\n\nn , \u03c3D\n\nn , \u03c3D\n\nn ), where \u03c3A\n\n4\n\n\f(A1) Q is a closed subset of M1(X ), and p /\u2208 Q.\n(A2) p(i) > 0 for all i \u2208 X . Furthermore, for each q \u2208 Q, q(i) > 0 for all i \u2208 X .\n(A3) c is continuous on Q, and there exists a unique q\u2217 \u2208 Q such that\n\nq\u2217 = arg min\nq\u2208Q\n\nc(q).\n\n(A4) The point p is distant from the set Q relative to the point q\u2217, i.e.,\n\nwhere D(\u00b5||\u03bd) =(cid:80)\n\nthe distributions \u00b5 and \u03bd.\n\n{\u00b5 \u2208 M1(X ) : D(\u00b5||p) \u2264 D(\u00b5||q\u2217)} \u2229 Q = \u2205,\n\ni\u2208X \u00b5(i) log \u00b5(i)\n\n\u03bd(i) , \u00b5, \u03bd \u2208 M1(X ), denotes the relative entropy between\n\nNote that (A1) and (A2) are very natural. In (A2), if p(i) = 0 for some i \u2208 X and q(i) > 0 for some\nq \u2208 Q, then the adversary will never pick q, as the defender can easily detect the presence of the\nattacker by looking for element i. On the other hand, if p(i) = 0 and q(i) = 0 for all q \u2208 Q, we\nmay consider a new alphabet set without i. In (A3), continuity of the cost function c is natural and\nwe do not assume any extra condition other than the requirement that there is a unique minimizer.\nAssumption (A4) is used to show certain property of the equilibrium of the defender, which is later\nused in the study of error exponents associated with classi\ufb01cation error. Speci\ufb01cally, Assumption (A4)\nis used in the proofs of Lemma 4.4, Lemma 4.5 and Theorem 4.1; all other results are valid without\nthis assumption. We will further discuss the role of (A3) and (A4) in Section 4.3 after Theorem 4.1.\n\n3.2 Model discussion\n\nOur setting is that of adversarial hypothesis testing, where the attacker chooses a distribution and\npoints are then generated i.i.d. according to it. This is a reasonable model in applications such\nas multimedia forensics (where one tries to determine if an attacker has tampered with an image\nfrom signals that can be modeled as random variables following an image-dependent distribution) or\nbiometrics (where again one tries to detect from random signals whether the perceived signals do\ncome from the characteristics of a given individual or they come from tampered characteristics)\u2014see\nmore details about these applications in [2, 3, 26]. In such applications, it is reasonable that different\nways of tampering have different costs for the attacker and that one can estimate those costs for a\ngiven application at least to some extent. Modeling the attacker\u2019s utility via a cost function is classical\nin other settings, for instance in adversarial classi\ufb01cation [12, 25, 6] and experiments with real-world\napplications where a reasonable cost function can be estimated has been done, for instance, in [6].\nOur setting is very similar to that of a composite hypothesis testing framework where nature picks a\ndistribution from a given set and generates independent samples from it. However, in such problems,\none does not model a utility function for the nature/statistician and one is often interested in existence\nand properties of uniformly most powerful test or locally most powerful test (depending on the\nBayesian or frequentist approach; see Section II.E in [23]). In contrast, here, we speci\ufb01cally model\nthe utility functions for the agents and investigate the behavior at Nash equilibrium using very\ndifferent analysis, which is more natural in adversarial settings where two rational agents interact.\nOur setting also coincides with the well-studied setting of minimax testing [15] when c(q) = 0 for\nall q \u2208 Q (and hence every q is a minimizer of c). Note, however, that this case is not included in\nour model due to Assumption (A3)\u2014rather we study the opposite extreme where c has a unique\nminimizer. Our results are not an easy extension of the classical results because our game is now a\nnonzero-sum game (whereas the minimax setting corresponds to a zero-sum game). We can therefore\nnot inherit any of the nice properties of zero-sum games; in particular we cannot compute the NE and\nwe instead have to prove properties of the NE (e.g., concentration) without being able to explicitly\ncompute it. In fact, our results too are quite different since we show that the error rate is the same as\na simple test where H1 would contain only q\u2217, which is different from the classical minimax case.\nFinally, in our model we \ufb01x the sample size n, i.e., the defender makes a decision only after\nobserving all n samples. We restrict to this simpler setting since it has applications in various domains\n(see Section 1.1), and understanding the equilibrium of such games leads to interesting and non-trivial\nresults. We leave the study of a sequential model where the defender has the \ufb02exibility to choose the\nnumber of samples for decision making as future work.\n\n5\n\n\f4 Main Results\n4.1 Mixed Strategy Nash Equilibrium for GB(d, n)\nWe \ufb01rst examine the Nash equilibrium for GB(d, n). Note that the strategy sets of both the attacker and\nthe defender are uncountable, hence it is a priori not clear whether our game has a Nash equilibrium.\nTowards this, we equip the set \u03a6n of all randomized decision rules with the sup-norm metric, i.e.,\n\ndn(\u03d51, \u03d52) = max\nxn\u2208X n\n\n|\u03d51(xn) \u2212 \u03d52(xn)|,\n\nfor \u03d51, \u03d52 \u2208 \u03a6n. It is easy to see that the set \u03a6n endowed with the above metric is a compact\nmetric space. We also equip M1(X ) with the usual Euclidean topology on Rd, and equip Q with the\nsubspace topology. Also, for studying the mixed extension of the game, we equip the spaces M1(Q)\nand M1(\u03a6n) with their corresponding weak topologies. Product spaces are always equipped with the\ncorresponding product topology.\nWe begin with a simple continuity property of the utility functions.\nLemma 4.1. Assume (A1)-(A3). Then, the utility functions uA\n\nn are continuous on Q \u00d7 \u03a6n.\nWe now show the main result of this subsection, namely existence and partial characterization of a\nNE for our hypothesis testing game.\nProposition 4.1. Assume (A1)-(A3). Then, there exists a mixed strategy Nash equilibrium for\nGB(d, n). If (\u02c6\u03c3A\n\nn , \u02c6\u03d5n) where \u02c6\u03d5n is the likelihood ratio test given by\n\nn ) is a NE, then so is (\u02c6\u03c3A\n\nn and uD\n\nn , \u02c6\u03c3D\n\n\uf8f1\uf8f2\uf8f3 1,\n\n,\n\n\u03d5\u02c6\u03c3D\n0,\n\nn\n\n(xn) \u2212 \u03b3p(xn) > 0,\n(xn) \u2212 \u03b3p(xn) = 0,\n(xn) \u2212 \u03b3p(xn) < 0,\n\nn\n\nif q\u02c6\u03c3A\nif q\u02c6\u03c3A\nif q\u02c6\u03c3A\n\n=(cid:82) \u03d5(xn)\u02c6\u03c3D\n\nn\n\nn\n\n\u02c6\u03d5n(xn) =\n\n(xn) =(cid:82) q(xn)\u02c6\u03c3A\n\n(4.1)\n\nn\n\nn\n\nn\n\nn (d\u03d5).\n\nn (dq), and \u03d5\u02c6\u03c3D\n\nwhere q\u02c6\u03c3A\nThe existence of a NE follows from Glicksberg\u2019s \ufb01xed point theorem (see e.g., Corollary 2.4 in [24]);\nfor the form of the defender\u2019s equilibrium strategy, we have to examine the utility function uD\nn .\nRemark 4.1. Note that we have considered randomization over \u03a6n to show existence of a NE. Once\nthis is established, we can then show the form of the strategy of the defender \u02c6\u03d5n at equilibrium; the\nexistence of a NE is not clear if we do not consider randomization over \u03a6n.\non X n cannot necessarily be written as an n-fold product\nRemark 4.2. Note that the distribution q\u02c6\u03c3A\ndistribution of some element from M1(X ). Therefore, the test \u02c6\u03d5n is slightly different from the\nusual likelihood ratio test that appears in the classical hypothesis testing problem where samples are\ngenerated independently.\nRemark 4.3. Apart from the conditions of the above proposition, a suf\ufb01cient condition for existence\nn (\u00b7, \u03d5) is\nof pure strategy Nash equilibrium is that the utilities are individually quasiconcave, i.e., uA\nn (q,\u00b7) is quasiconcave for all q \u2208 Q. However, it is easy to check\nquasiconcave for all \u03d5 \u2208 \u03a6n, and uD\nthat the Type II error term is not quasiconcave in the attacker\u2019s strategy, and hence the utility of the\nattacker is not quasiconcave. Hence, a pure strategy Nash equilibrium is not guaranteed to exist\u2014see\nnumerical experiments in Appendix C of the full version of this paper [27].\nRemark 4.4. Proposition 4.1 does not provide any information about the structure of the attacker\u2019s\nstrategy at a NE. We believe that obtaining the complete structure of a NE and computing it is a\ndif\ufb01cult problem in general because the strategy spaces of both players are uncountable (and there is\nno pure-strategy NE in general), and we cannot use the standard techniques for \ufb01nite games. However,\nwe emphasize that we are able to obtain error exponents at an equilibrium (see Theorem 4.1) without\nexplicitly computing the structure of a NE. Also, one could study Stackelberg equilibrium for our\ngame GB(d, n) to help solve computational issues, although we note that most of the security games\nliterature using Stackelberg games assumes \ufb01nite action spaces (see, for example, [17]); however we\ndo not address the study of Stackelberg equilibrium in this paper.\n\n4.2 Concentration Properties of Equilibrium\n\nWe now study some concentration properties of the mixed strategy Nash equilibrium for the game\nGB(d, n) for large n. The results in this section will be used later to show the exponential convergence\nof the classi\ufb01cation error at equilibrium.\n\n6\n\n\fn , \u02c6\u03c3D\nn )\n\nn , \u02c6\u03c3D\n\nn , \u02c6\u03c3D\n\nn ) \u2192 0 as n \u2192 \u221e.\n\nn (q, \u03d5), q \u2208 Q, \u03d5 \u2208 \u03a6n. We begin with\n\nLet en denote the classi\ufb01cation error, i.e., en(q, \u03d5) = \u2212uD\nthe following lemma, which asserts that the error at equilibrium is small for large enough n.\nn )n\u22651 be a sequence such that, for each n \u2265 1, (\u02c6\u03c3A\nLemma 4.2. Assume (A1)-(A3). Let (\u02c6\u03c3A\nis a mixed strategy Nash equilibrium for GB(d, n). Then, en(\u02c6\u03c3A\nThe main idea in the proof is to let the defender play a decision rule whose acceptance set is a small\nneighborhood around the point p, and then bound en(\u02c6\u03c3A\nn ) using the error of the above strategy.\nn , \u02c6\u03c3D\nWe now show that the mixed strategy pro\ufb01le of the attacker \u02c6\u03c3A\nn converges weakly to the point mass at\nq\u2217 (denoted by \u03b4q\u2217) as n \u2192 \u221e. This is a consequence of the fact that q\u2217 is the minimizer of c, and\nhence for large enough n, the attacker does not gain much by deviating from the point q\u2217.\nn \u2192 \u03b4q\u2217 weakly\nLemma 4.3. Assume (A1)-(A3), and let (\u02c6\u03c3A\nas n \u2192 \u221e.\nNote that it is not clear from the above lemma that the equilibrium strategy of the attacker \u02c6\u03c3A\nn is\nsupported on a small neighborhood around q\u2217 for large enough n. By playing a strategy q that is far\nfrom q\u2217 we could still have uA\nn could compensate\nfor the possible loss of utility from the cost term. We now proceed to show that this cannot happen\nunder Assumption (A4). We \ufb01rst argue that the equilibrium error is small even when the attacker\ndeviates from her equilibrium strategy.\nLemma 4.4. Assume (A1)-(A4), and let (\u02c6\u03c3A\n\nn )n\u22651 be as in Lemma 4.2. Then, \u02c6\u03c3A\n\nn ), since the error term in uA\n\nn (q, \u02c6\u03c3D\n\nn ) = uA\n\nn (\u02c6\u03c3A\n\nn , \u02c6\u03c3D\n\nn , \u02c6\u03c3D\n\nn , \u02c6\u03c3D\n\nn )n\u22651 be as in Lemma 4.2. Then,\nn ) \u2192 0 as n \u2192 \u221e.\n\nen(q, \u02c6\u03c3D\n\nsup\nq\u2208Q\n\nn , \u02c6\u03c3D\n\nn )n\u22651 be as in Lemma 4.2. Let (qn)n\u22651 be a sequence\n\nn ) for each n \u2265 1. Then, qn \u2192 q\u2217 as n \u2192 \u221e.\n\nWe are now ready to show the concentration of the attacker\u2019s equilibrium:\nLemma 4.5. Assume (A1)-(A4), and let (\u02c6\u03c3A\nsuch that qn \u2208 supp(\u02c6\u03c3A\nThe above concentration phenomenon is a consequence of the uniqueness of q\u2217 and Assumption (A4).\nThe main idea in the proofs of Lemma 4.4 and Lemma 4.5 is to essentially show that, for large enough\nn, the acceptance region of H0 under (any) mixed strategy Nash equilibrium does not intersect the set\nQ. If we do not assume (A4), then the decision region at equilibrium could intersect Q, and we may\nnot have the concentration property in the above lemma (we will still have the convergence property\nin Lemma 4.3 though, which does not use (A4)).\n\n4.3 Error Exponents\n\nWith the results on concentration properties of the equilibrium from the previous section, we are\nnow ready to examine the error exponent associated with the classi\ufb01cation error at equilibrium. Let\n\u039b0 denote the log-moment generating function of the random variable q\u2217(X)\np(X) under H0, i.e., when\n\np(i), \u03bb \u2208 R. De\ufb01ne its Fenchel-Legendre transform\n\nX \u223c p: \u039b0(\u03bb) = log(cid:80)\n\n(cid:16)\n\n(cid:17)\n\ni\u2208X exp\n\np(i)\n\n\u03bb q\u2217(i)\n\u039b\u2217\n0(x) = sup\n\u03bb\u2208R\n\n{\u03bbx \u2212 \u039b0(\u03bb)}, x \u2208 R.\n\nOur main result in the paper (for the Bayesian case) is the following theorem.\nTheorem 4.1. Assume (A1)-(A4), and let (\u02c6\u03c3A\n\nn )n\u22651 be as in Lemma 4.2. Then,\n\nn , \u02c6\u03c3D\n\nlim\nn\u2192\u221e\n\n1\nn\n\nlog en(\u02c6\u03c3A\n\nn , \u02c6\u03c3D\n\nn ) = \u2212\u039b\u2217\n\n0(0).\n\nn , \u02c6\u03c3D\n\nOur approach to show this result is via obtaining asymptotic lower and upper bounds for the classi\ufb01-\ncation error at equilibrium en(\u02c6\u03c3A\nn ). Since we do not have much information about the structure\nof the equilibrium, we \ufb01rst let one of the players deviate from their equilibrium strategy, so that we\ncan estimate the error corresponding to the new pair of strategies, and then use these estimates to\ncompute the error rate at equilibrium. The lower bound easily follows by letting the attacker play the\nstrategy q\u2217, and using the error exponent in the classical hypothesis testing problem between p and\nq\u2217. For the upper bound, we let the defender play a speci\ufb01c deterministic decision rule, and make use\nof the concentration properties of the equilibrium of the attacker in Section 4.2.\n\n7\n\n\fThus, we see that the error exponent is the same as that for the classical hypothesis testing problem of\nX1, . . . , Xn i.i.d.\u223c p versus X1, . . . , Xn i.i.d.\u223c q\u2217 (see Corollary 3.4.6 in [10]). That is, for large\nvalues of n, the adversarial hypothesis testing game is not much different from the above classical\nsetting (whose parameters are derived from the adversarial setting) in terms of classi\ufb01cation error.\nWe emphasize that we have not used any property of the speci\ufb01c structure of the mixed strategy\nNash equilibrium in obtaining the error exponent associated with the classi\ufb01cation error, and hence\nTheorem 4.1 is valid for any NE. We believe that obtaining the actual structure of a NE is a dif\ufb01cult\nproblem, as the strategy spaces are in\ufb01nite, and the utility functions do not possess any monotonicity\nproperties in general. For numerical computation of error exponents in a simple case, see Section 5.\nWe conclude this section by discussing the role of Assumptions (A3) and (A4). We used (A4) to obtain\nthe concentration of equilibrium in Lemma 4.5. Without this assumption, Theorem 4.1 is not valid; see\nSection 5 for numerical counter-examples. Also, in our setting, unlike the classical minimax testing,\nit is not clear whether it is always true that the error goes to 0 as the number of samples becomes\nlarge, and whether the attacker should always play a point close to q\u2217 at equilibrium. It could be that\nplaying a point far from q\u2217 is better if she can compensate the loss from c from the error term. In fact,\nthat is what happens when (A4) is not satis\ufb01ed, since there is partial overlap of the decision region of\nthe defender with the set Q. Regarding (A3), when c has multiple minimizers, our analysis can only\ntell us that the equilibrium of the attacker is supported around the set of minimizers for large enough\nn; to study error exponents in such cases, one has to do a \ufb01ner analysis of characterizing the attacker\u2019s\nequilibrium. All in all, using (A3) and (A4) allows us to establish interesting concentration properties\nof the equilibrium (which is not a priori clear) and error exponents associated with classi\ufb01cation error\nwithout characterizing a NE, hence we believe that these assumptions serve as a good starting point.\n\n5 Numerical Experiments\n\nIn this section, we present two numerical examples in the Bayesian formulation to illustrate the\nresult in Theorem 4.1 and the importance of Assumption (A4). Due to space limitations, additional\nexperiments in the Bayesian formulation are relegated to Appendix C of the full version [27], which\nillustrate (a) best response strategies of the players, (b) existence of pure strategy Nash equilibrium\nfor large values on n as suggested by Lemma 4.5, and (c) importance of Assumption (A4).1\nWe illustrate the result in Theorem 4.1 numerically in the following setting. We \ufb01x X = {0, 1} (i.e.\nd = 2) and each probability distribution on X is represented by the probability that it assigns to the\nsymbol 1, and hence M1(X ) is viewed as the unit interval. We \ufb01x p = 0.5. For numerical compu-\ntations, we discretize the set Q into 100 equally spaced points, and we only consider deterministic\nthreshold-based decision rules for the defender. To compute a NE, we solve the linear programs\nassociated with attacker as well as the defender for the zero-sum equivalent game of GB(2, n).\nFor the function c(q) = |q \u2212 q\u2217| with q\u2217 = 0.8, Figure 1(a) shows the error exponent at the\nNE computed by the above procedure as a function of the number of samples, from n = 10 to\nn = 300 in steps of 10. As suggested by Theorem 4.1, we see that the error exponents approach\nthe value \u039b\u2217\n0(0) = 0.054 (the boundary of the decision region is around the point q = 0.66, and\nD(q||p) \u2248 D(q||q\u2217) \u2248 0.054).\nWe now consider an example which demonstrates that, the result on error exponent in Theorem 4.1\nmay not be valid if Assumption (A4) is not satis\ufb01ed. In this experiment, we consider the case\nwhere Q = [0.6, 0.9] and q\u2217 = 0.9. Note that the present setting does not satisfy Assumption (A4).\nFigure 1(b) shows the error exponent at the equilibrium as a function of n, from n = 100 to n = 400\nin steps of 100, for the cost function c(q) = 3|q \u2212 q\u2217|. From this plot, we see that, the error exponents\nconverge to somewhere around 0.032, whereas \u039b\u2217\n\n0(0) \u2248 0.111.\n\n6 Concluding Remarks\n\nIn this paper, we studied hypothesis testing games that arise in the context of adversarial classi\ufb01cation.\nWe showed that, at equilibrium, the strategy of the classi\ufb01er is to use a likelihood ratio test. We also\n\n1Appendix C of the full version [27] also contains numerical experiments in the Neyman-Pearson formulation\npresented in Appendix A of the full version [27]. The code used for our simulations is available at https:\n//github.com/sarath1789/ahtg_neurips2019.\n\n8\n\n\f(a) Q = [0.7, 0.9], c(q) = |q \u2212 0.8|\n\n(b) Q = [0.6, 0.9], c(q) = 3|q \u2212 0.9|\n\nFigure 1: Error exponents as a function of n\n\nexamined the exponential rate of decay of classi\ufb01cation error at equilibrium and showed that it is\nsame as that of a classical testing problem with parameters derived from the adversarial model.\nThroughout the paper, we assumed that the alphabet X is \ufb01nite. This is a reasonable assumption\nin applications that deal with digital signals such as image forensics (an important application for\nadversarial hypothesis testing); and it is also a good starting point because even in this case, our\nanalysis of the error exponents is nontrivial. Making X countable/uncountable will make the space\nM1(X ) in\ufb01nite dimensional, and the analysis of error exponents will become more dif\ufb01cult (e.g., the\ncontinuity of relative entropy is no longer true in this case, which we crucially use in our analysis),\nbut the case of general state space X is an interesting future direction.\nFinding the exact structure of the equilibrium for our hypothesis testing games is a challenging\nfuture direction. This will also shed some light on the error exponent analysis for the case when\nAssumption (A4) is not satis\ufb01ed. Another interesting future direction is to examine the hypothesis\ntesting game in the sequential detection context where the defender can also decide the number of\ndata samples for classi\ufb01cation. In such a setting, an important question is to understand whether the\noptimal strategy of the classi\ufb01er is to use a standard sequential probability ratio test.\n\nAcknowledgments\n\nThe \ufb01rst author is partially supported by the Cisco-IISc Research Fellowship grant. The work of the\nsecond author was supported in part by the French National Research Agency (ANR) through the\n\u201cInvestissements d\u2019avenir\u201d program (ANR-15-IDEX-02) and through grant ANR-16- TERC0012;\nand by the Alexander von Humboldt Foundation.\n\nReferences\n[1] Bao, N., Kreidl, P., and Musacchio, J. (2011). Binary hypothesis testing game with training data. In\nProceedings of the 2nd International Conference on Game Theory for Networks (GameNets), pages 265\u2013280.\n\n[2] Barni, M. and Tondi, B. (2013). The source identi\ufb01cation game: An information-theoretic perspective.\n\nIEEE Transactions on Information Forensics and Security, 8(3):450\u2013463.\n\n[3] Barni, M. and Tondi, B. (2014). Binary hypothesis testing game with training data. IEEE Transactions on\n\nInformation Theory, 60(8):4848\u20134866.\n\n[4] Barreno, M., Nelson, B., Joseph, A. D., and Tygar, J. D. (2010). The security of machine learning. Machine\n\nLearning, 81(2):121\u2013148.\n\n[5] Brand\u00e3o, F. G. S. L., Harrowy, A. W., Leez, J. R., and Peres, Y. (2014). Adversarial hypothesis testing and a\nquantum stein\u2019s lemma for restricted measurements. In Proceedings of the 5th Innovations in Theoretical\nComputer Science conference (ITCS), pages 183\u2013194.\n\n[6] Br\u00fcckner, M., Kanzow, C., and Scheffer, T. (2012). Static prediction games for adversarial learning\n\nproblems. The Journal of Machine Learning Research, 13(1):2617\u20132654.\n\n9\n\n\f[7] Br\u00fcckner, M. and Scheffer, T. (2011). Stackelberg games for adversarial prediction problems. In Proceedings\nof the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD),\npages 547\u2013555.\n\n[8] Chen, L. and Leneutre, J. (2009). A game theoretical framework on intrusion detection in heterogeneous\n\nnetworks. IEEE Transactions on Information Forensics and Security, 4(2):165\u2013178.\n\n[9] Dalvi, N., Domingos, P., Mausam, Sanghai, S., and Verma, D. (2004). Adversarial classi\ufb01cation. In\nProceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining\n(KDD), pages 99\u2013108.\n\n[10] Dembo, A. and Zeitouni, O. (2010). Large Deviations: Techniques and Applications. Springer-Verlag,\n\nBerlin Heidelberg, 2nd edition.\n\n[11] Dong, J., Roth, A., Schutzman, Z., Waggoner, B., and Wu, Z. S. (2018). Strategic classi\ufb01cation from\nrevealed preferences. In Proceedings of the 2018 ACM Conference on Economics and Computation (EC),\npages 55\u201370.\n\n[12] Dritsoula, L., Loiseau, P., and Musacchio, J. (2017). A game-theoretic analysis of adversarial classi\ufb01cation.\n\nIEEE Transactions on Information Forensics and Security, 12(12):3094\u20133109.\n\n[13] Hardt, M., N, M., Papadimitriou, C., and Wootters, M. (2016). Strategic classi\ufb01cation. In Proceedings of\n\nthe 7th Innovations in Theoretical Computer Science conference (ITCS), pages 111\u2013122.\n\n[14] Huang, L., Joseph, A. D., Nelson, B., Rubinstein, B. I. P., and Tygar, J. D. (2011). Adversarial machine\nlearning. In Proceedings of the 2011 ACM Workshop on Arti\ufb01cial Intelligence and Security (AISec), pages\n43\u201358.\n\n[15] Ingster, Y. I. and Suslina, I. A. (2003). Nonparametrie Goodness-of-Fit Testing Under Gaussian Models,\n\nvolume 169 of Lecture Notes in Statistics. Springer-Verlag, New York.\n\n[16] Kantarcioglu, M., Xi, B., and Clifton, C. (2011). Classi\ufb01er evaluation and attribute selection against 344\n\nactive adversaries. Data Mining and Knowledge Discovery, 22(1):291\u2013335.\n\n[17] Korzhyk, D., Yin, D., Kiekintveld, C., Conitzer, V., and Tambe, M. (2011). Stackelberg vs. nash in security\ngames: An extended investigation of interchangeability, equivalence, and uniqueness. Journal of Arti\ufb01cial\nIntelligence Research, 41(2):297\u2013327.\n\n[18] Li, B. and Vorobeychik, Y. (2015). Scalable optimization of randomized operational decisions in adversarial\nclassi\ufb01cation settings. In Proceedings of the Eighteenth International Conference on Arti\ufb01cial Intelligence\nand Statistics (AISTATS), pages 599\u2013607.\n\n[19] Lis\u00fd, V., Kessl, R., and Pev\u00fd, T. (2014). Randomized operating point selection in adversarial classi\ufb01cation.\nIn Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge\nDiscovery in Databases (ECML PKDD), pages 240\u2013255.\n\n[20] Lowd, D. and Meek, C. (2005). Adversarial learning.\n\nIn Proceedings of the 11th ACM SIGKDD\n\nInternational Conference on Knowledge Discovery in Data Mining (KDD), pages 641\u2013647.\n\n[21] Lye, K. and Wing, J. M. (2005). Game strategies in network security. International Journal of Information\n\nSecurity, 4(1-2):71\u201386.\n\n[22] Papernot, N., McDaniel, P., Sinha, A., and Wellman, M. (2018). Towards the science of security and\nprivacy in machine learning. In Proceedings of the 3rd IEEE European Symposium on Security and Privacy.\n\n[23] Poor, H. (1994). An Introduction to Signal Detection and Estimation. Springer-Verlag, New York, 2nd\n\nedition.\n\n[24] Reny, P. J. (2005). Non-cooperative games: Equilibrium existence. In The New Palgrave Dictionary of\n\nEconomics. Palgrave Macmillan.\n\n[25] Soper, B. and Musacchio, J. (2015). A non-zero-sum, sequential detection game. In Proceedings of the\n\nAllerton Conference on Communication, Control, pages 361\u2013371.\n\n[26] Tondi, B., Merhav, N., and Barni, M. (2019). Detection games under fully active adversaries. Entropy,\n\n21(1):23.\n\n[27] Yasodharan, S. and Loiseau, P. (2019). Nonzero-sum adversarial hypothesis testing games. Full version,\n\navailable at https://hal.inria.fr/hal-02299451.\n\n10\n\n\f[28] Zhou, Y. and Kantarcioglu, M. (2014). Adversarial learning with bayesian hierarchical mixtures of experts.\n\nIn Proceedings of the 2014 SIAM International Conference on Data Mining (SDM), pages 929\u2013937.\n\n[29] Zhou, Y., .Kantarcioglu, M., Thuraisingham, B., and Xi, B. (2012). Adversarial support vector machine\nlearning. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and\nData Mining (KDD), pages 1059\u20131067.\n\n11\n\n\f", "award": [], "sourceid": 3972, "authors": [{"given_name": "Sarath", "family_name": "Yasodharan", "institution": "Indian Institute of Science"}, {"given_name": "Patrick", "family_name": "Loiseau", "institution": "Inria"}]}