{"title": "Best Response Regression", "book": "Advances in Neural Information Processing Systems", "page_first": 1499, "page_last": 1508, "abstract": "In a regression task, a predictor is given a set of instances, along with a real value for each point. Subsequently, she has to identify the value of a new instance as accurately as possible. In this work, we initiate the study of strategic predictions in machine learning.  We consider a regression task tackled by two players, where the payoff of each player is the proportion of the points she predicts more accurately than the other player. We first revise the probably approximately correct learning framework to deal with the case of a duel between two predictors.  We then devise an algorithm which finds a linear regression predictor that is a best response to any (not necessarily linear) regression algorithm. We show that it has linearithmic sample complexity, and polynomial time complexity when the dimension of the instances domain is fixed. We also test our approach in a high-dimensional setting, and show it significantly defeats classical regression algorithms in the prediction duel. Together, our work introduces a novel machine learning task that lends itself well to current competitive online settings, provides its theoretical foundations, and illustrates its applicability.", "full_text": "Best Response Regression\n\nTechnion - Israel Institute of Technology\n\nTechnion - Israel Institute of Technology\n\nOmer Ben-Porat\n\nHaifa 32000 Israel\n\nMoshe Tennenholtz\n\nHaifa 32000 Israel\n\nomerbp@campus.technion.ac.il\n\nmoshet@ie.technion.ac.il\n\nAbstract\n\nIn a regression task, a predictor is given a set of instances, along with a real value\nfor each point. Subsequently, she has to identify the value of a new instance as\naccurately as possible. In this work, we initiate the study of strategic predictions in\nmachine learning. We consider a regression task tackled by two players, where the\npayoff of each player is the proportion of the points she predicts more accurately\nthan the other player. We \ufb01rst revise the probably approximately correct learning\nframework to deal with the case of a duel between two predictors. We then devise\nan algorithm which \ufb01nds a linear regression predictor that is a best response to\nany (not necessarily linear) regression algorithm. We show that it has linearithmic\nsample complexity, and polynomial time complexity when the dimension of the\ninstances domain is \ufb01xed. We also test our approach in a high-dimensional setting,\nand show it signi\ufb01cantly defeats classical regression algorithms in the prediction\nduel. Together, our work introduces a novel machine learning task that lends itself\nwell to current competitive online settings, provides its theoretical foundations, and\nillustrates its applicability.\n\n1\n\nIntroduction\n\nPrediction is fundamental to machine learning and statistics. In a prediction task, an algorithm is\ngiven a sequence of examples composed of labeled instances, and its goal is to learn a general rule\nthat maps instances to labels. When the labels take continuous values, the task is typically referred\nto as regression. The quality of a regression algorithm is measured by its success in predicting the\nvalue of an unlabeled instance. Literature on regression is mostly concerned with minimizing the\ndiscrepancy of the prediction, i.e. the difference between the true value and the predicted one.\nDespite the tremendous amount of work on prediction and regression, online commerce presents new\nchallenges. In this context, prediction is not carried out in isolation. New entrants can utilize knowl-\nedge of previous expert predictions and the corresponding true values, to maximize their probability\nof predicting better than that expert, treated as the new entrant\u2019s opponent. This fundamental task is\nthe main challenge we tackle in this work.\nWe initiate the study of strategic predictions in machine learning. We present a regression learning\nsetting that stems from a game-theoretic point of view, where the goal of the learner is to maximize\nthe probability of being the most accurate among a set of predictors. Note that this approach may be\nin con\ufb02ict with the traditional prediction goal.\nConsider an online real estate expert, who frequently predicts the sale value of apartments. This\nexpert, having been in the market for a while, has historical data on the values and characteristics\nof similar apartments. For simplicity, assume the expert uses simple linear regression to predict the\nvalue of an apartment as a function of its size. When a new apartment comes on the market, the\nexpert uses her gathered historical data to predict the new apartment\u2019s value. When the apartment is\nsold, the true value (and the accuracy of the prediction) is revealed.\n\n31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA.\n\n\fFigure 1: A case where minimizing the square error can be easily beaten. Each point is an instance-\nvalue pair, where the circles are historical points (i.e. their value has been revealed) and the triangles\nare new points, unseen by either the expert and the agent. The red (solid) line represents the linear\nleast squares estimators, employed by the expert. After collecting a suf\ufb01cient amount of historical\ndata (circles) on apartments along with their true value and the value predicted by the expert, the\nagent comes up with the response represented by the green (dashed) line. For each of the unseen\napartment sizes, both the expert and the agent declare their predictions of the apartment\u2019s value.\nNotice that the agent outperforms the expert in the majority of the historical points. In addition, the\nagent produces a more accurate prediction in the majority of the new (unseen) points.\n\nAt \ufb01rst glance this seems extremely effective, however it is also extremely fragile. An agent who\nenters the real estate business may come up with a linear predictor for which the probability (over\nall apartments and their values) of being more accurate is high, making it the preferable predictor.\nFigure 1 illustrates our approach. The expert uses linear least square estimators (LSE) to minimize\nthe mean square error (MSE). The agent, after having collected \"enough\" historical data (circles) and\nhaving observed the predictions of the expert, produces a strategy (regression line). Both the expert\nand the agent predict the value of new apartments coming on the market (triangles). As illustrated,\nthe prediction of the agent is the most accurate in the majority of new instances.\nOne criticism of this novel approach is that while maximizing the probability of being the most\naccurate, the agent may produce \"embarrassing\" predictions for some instances. Current prediction\nalgorithms are designed to minimize some measure of overall loss, such as the MSE. Notice that in\nmany, and perhaps even most, practical scenarios, being a better predictor on more instances is more\nimportant than avoiding such sporadic \"embarrassing predictions\". In particular, our approach \ufb01ts\nany commerce and advertising setting where the agent offers predictions to users on the value of\ndifferent goods or services, aiming at maximizing the number of users that will \ufb01nd her predictions\nmore accurate than the one provided by the expert. For example, an agent, serving users searching\nfor small apartments, would be happy to fail completely in predicting the value of very large sized\napartments if this allowed predicting the value of smaller apartments better than an opponent.\nOur novel perspective suggests several new fundamental problems:\n\n1. Given a prediction algorithm ALG (e.g. LSE), what would be the best response to ALG, if\nwe aim at maximizing the probability that the new algorithm would be more accurate than\nALG?\n\n2. In case ALG is unknown, but the agent has access to a labeled set of instances along with\nthe prediction made by ALG for each instance, how many i.i.d. samples are needed in order\nto learn a best response to ALG over the whole population?\n\n3. How poorly do classical regression algorithms preform against such a best response algo-\n\nrithm?\n\nIn this work, we focus on a two player scenario and analyze the best response of the agent against an\nopponent. We examine the agent\u2019s perspective, and introduce a rigorous treatment of Problems 1-3\nabove. We model the task of \ufb01nding a best response as a supervised learning task, and show that it\n\n2\n\nSize(squarefeet)Value(dollars)ExpertAgent\f\ufb01ts the probably approximately correct (PAC) learning framework. Speci\ufb01cally, we show that when\nthe strategy space of the agent is restricted, a best response over a large enough sample set is likely to\nbe an approximate best response over the unknown distribution.\nOur main result deals with an agent employing linear regression in Rn for any constant n. We present\na polynomial time algorithm which computes a linear best response (i.e. from the set of all linear\npredictors) to any regression algorithm employed by the opponent. We also show a linearithmic\nbound in the number of training samples needed in order to successfully learn a best response. In\naddition, we show that in some cases our algorithm can be adapted to have an MSE score arbitrarily\nclose to that of the given regression algorithm ALG. The theoretical analysis is complemented by\nan experimental study, which illustrates the effectiveness of our approach. In order to \ufb01nd a best\nlinear response in high dimensional space, we provide a mixed integer linear programming (MILP)\nalgorithm. The MILP algorithm is tested on the Boston housing dataset [5]. Indeed, we show that we\ncan outperform classical regression algorithms in up to 70% of the points. Moreover, we outperform\nclassical regression algorithms even in the case where they have full access to both training and test\ndata, while we restrict our responder algorithm to the use of the training data only.\n\nOur contribution. Our contributions are 3-fold. The main conceptual contribution of this paper is\nthe explicit suggestion that a prediction task may have strategic aspects. We introduce the setting\nof best response regression, applicable to a huge variety of scenarios, and revise the PAC-learning\nframework to deal with such a duel framework. Then, we show an ef\ufb01cient algorithm dealing with\n\ufb01nding a best-response linear regression in Rn for any constant n, against any regression algorithm.\nThis best response algorithm maximizes the probability of beating the latter on new instances. Finally,\nwe present an experimental study showing the applicability of our approach. Together, this work\noffers a new machine learning challenge, addresses some of its theoretical properties and algorithmic\nchallenges, while also showing its applicability.\n\n1.1 Related work\n\nThe intersection of learning theory with multi-agent systems is expanding with the rise of data science.\nIn the \ufb01eld of mechanism design [8], [3, 7] considered prediction tasks with strategic aspects. In their\nmodel, the instances domain is to be labeled by one agent, and the dataset is constructed of points\ncontrolled by sel\ufb01sh users, who have their own view on how to label the instances domain. Hence,\nthe users can misreport the points in order to sway decisions in their favor. A different line of work\nthat is related to our model is the analysis of sample complexity in revenue maximizing auctions. In\na recent work [2] the authors reconsider an auction setting where the auctioneer can sample from\nthe valuation functions of the bidders, thereby relaxing the ubiquitous assumption of knowing the\nunderlying distribution over bidders\u2019 valuations.\nWhile the above papers consider mechanism design problems inspired by machine learning, our work\nconsiders a novel machine learning problem inspired by game theory.\nIn work on dueling algorithms [6], an optimization problem is analyzed from the perspective of\ncompetition, rather than from the point of view of a single optimizer. That work examines the dueling\nform of several optimization problems, e.g. the shortest path from the source vertex to the target\nvertex in a graph with random weights. While minimizing the expected length is a probable solution\nconcept for a single optimizer, this is no longer the case in the de\ufb01ned duel. While [6] assumes a\ncommonly-known distribution over a \ufb01nite set of instances, we have no such assumption. Instead,\nwe consider a sample set drawn from the underlying distribution with the aim of predicting a new\ninstance better than the opponent.\nOur formulation is also related to the Learning Using Privileged Information paradigm (see, e.g.,\n[9, 14, 15]), in which the learner (agent) is supplied with additional information along with the label\nof each instance. In this paper, we assume the agent has access to predictions made by another\nalgorithm (the opponent\u2019s), which can be treated as additional information.\n\n2 Problem formulation\n\nThe environment is composed of instances and labels. In the motivating example given above, the\ninstances are the characteristics of the apartments, and the labels are the values of these apartments.\n\n3\n\n\fA set of N players offer predictive services, where a strategy of a player is a labeling function.\nFor each instance-label pair (x, y), the players see x, and subsequently each player i, predicts\nthe value of the y. We call this label estimate \u02c6yi. The player who wins a point (x, y) is the one\nwith the smallest discrepancy, i.e. mini | \u02c6yi \u2212 y|. Under the strategy pro\ufb01le (h1, . . . hN ), where\neach entry is the labeling function chosen by the corresponding player, the payoff of Player i is\nPr ({(x, y) : Player i wins (x, y)}).\nA strategy of a player is called a best response if it maximizes the payoff of that player, when the\nstrategies of all the other players are \ufb01xed. In this work, we analyze the best response of a player, and\nw.l.o.g. we assume she has only one opponent. The model is as follows:\n\n1. We assume a distribution over the examples domain, which is the cross product of the\n\n2. The agent and the opponent both predict the label of each instance. The opponent uses a\n\ninstances domain X \u2282 Rn and the labels domain Y \u2282 R.\nstrategy \u00afh, which is a conditional distribution over R given x \u2208 X .\n3. The agent is unaware of the distribution over X \u00d7 Y or the strategy of the opponent \u00afh.\nHence, we explicitly address the joint distribution D over Z = X \u00d7 Y \u00d7 R, where a triplet\n(x, y, p) represents an instance x, its label y, and the discrepancy of the opponent\u2019s predicted\nvalue p, i.e. p = |\u00afh(x) \u2212 y|. We stress that D is unknown to the agent.\n\n4. The payoff of the agent under the strategy h : X \u2192 Y is given by\n\n(cid:0)1\n\n(cid:1) .\n\n\u03c0D(h) =\n\nE\n\n(x,y,p)\u223cD\n\n|h(x)\u2212y)|<p\n\n5. The agent has access to a sequence of examples S, with which she wishes to maximize her\n\npayoff.\n\nNote that a strategy which outputs yi for every instance xi in S may look promising, but will probably\nlead to over\ufb01tting, and low payoff for the agent. Since the agent wishes to generalize from S to D,\nrestricting the strategy set to H \u2282 YX seems justi\ufb01ed. We de\ufb01ne the goal of the agent:\n\n6. The agent is willing to restrict herself to a strategy from H \u2282 YX . Her goal: to \ufb01nd an\nalgorithm which, given \u0001, \u03b4 \u2208 (0, 1) and a sequence of m = m(\u0001, \u03b4) examples S sampled\ni.i.d. from D, outputs a strategy h\u2217 such that with probability at least 1 \u2212 \u03b4 (over the choices\nof S) it holds that\n\n\u03c0D(h\u2217) \u2265 sup\nh\u2208H\n\n\u03c0D(h) \u2212 \u0001.\n\nIndeed, the access to a sequence of examples seems realistic, and the size of S depends on the amount\nof resources at the agent\u2019s disposal. The size of S also affects the selection of H: if the agent can\ngather \"many\" examples, she might be able to learn a \"good\" strategy from a more complex strategy\nspace.\nWe say that h \u2208 H is an approximate best response with factor \u0001 if for all h(cid:48) \u2208 H it holds that\n\u03c0D(h(cid:48)) \u2212 \u03c0D(h) \u2264 \u0001. Note that the goal of the agent can be interpreted as \ufb01nding an approximate\nbest response with high probability. The empirical payoff of the agent is de\ufb01ned by\n\n(cid:12)(cid:12){i : 1\n\n\u03c0S (h) =\n\n1\nm \u00b7\n\n|h(xi)\u2212yi)|<pi}\n\n(cid:12)(cid:12) ,\n\nand a strategy h \u2208 arg maxh(cid:48)\u2208H \u03c0S (h(cid:48)) is called an empirical best response (w.r.t S). Next, we\nadopt the PAC framework [12] to de\ufb01ne under which strategy spaces an empirical best response is\nlikely to be an approximate best response.\n\n2.1 Approximate best response with PAC learnability\n\nThe \ufb01eld of statistical learning addresses the problem of \ufb01nding a predictive function based on data.\nWe brie\ufb02y de\ufb01ne some key concepts in learning theory, that will be used later. For a more gentle\nintroduction the reader is referred to [11].\nLet G be a class of functions from Z to {0, 1} and let S = {z1, . . . , zm} \u2282 Z. The restriction of\nG to S, denoted G(S), is de\ufb01ned by G(S) = {(g(z1), g(z2), . . . , g(zm)) : g \u2208 G}. Namely, G(S)\ncontains all the binary vectors induced by the functions in G on the items of S. We say that G shatters\nS if G(S) contains all binary vectors of size m, i.e. |G(S)| = 2m.\n\n4\n\n\fDe\ufb01nition 1 (VC dimension,[13]). The VC dimension of a class G, denoted VCdim(G), is the\nmaximal size of a set S \u2282 Z that can be shattered by G.\nDe\ufb01nition 2 (PAC learnability,[12]). A hypothesis class H is PAC-learnable with respect to a domain\nset Z and a loss function l : H \u00d7 Z \u2192 R+ , if there exists a function \u03c4H : (0, 1)2 \u2192 N and a\nlearning algorithm ALG such that for every \u0001, \u03b4 \u2208 (0, 1) and for every distribution D over Z, when\nrunning ALG on m \u2265 \u03c4H(\u0001, \u03b4) i.i.d. examples generated by D, it returns a hypothesis h \u2208 H such\nthat with probability of at least 1 \u2212 \u03b4 it holds that\n(1)\nLD(h) \u2264 inf\nh(cid:48)\u2208H\n\nLD(h(cid:48)) + \u0001,\n\nwhere LD(h) = Ez\u223cZ l(h, z).\nLet H be a class of functions from X to Y, and let Z = X \u00d7 Y \u00d7 R, as de\ufb01ned earlier in this section.\nTypically in a regression task, the hypothesis class is restricted in order to decrease the distance\nbetween the predicted labels and the true label. In the aforementioned model, however, the agent may\nwant to deliberately harm her accuracy on some subset of the instances domain. She will do this as\nlong as it increases the number of instances having a better prediction, thereby improving her payoff.\nSince h \u2208 H can either win a point (x, y, p) or lose it, the model resembles a binary classi\ufb01cation\ntask, where the \"label\" of (x, y, p) is the identity of the winner. That is, a triplet (x, y, p) would be\nlabeled 1 if the agent produced a better prediction than the opponent, and zero otherwise. However,\nnotice that the agent\u2019s strategy is involved in the labeling. This is, of course, not the case of binary\nclassi\ufb01cation. Our approach is to introduce a corresponding binary classi\ufb01cation problem, and by\nleveraging former results obtained on binary classi\ufb01cation, deduce suf\ufb01cient learnability conditions\nfor our model. The complete reduction is described in detail in the appendix.\nAdjusting to the loss function framework, de\ufb01ne:\n\n\u2200z = (x, y, p) \u2208 Z : l(h, z) =\n\n|h(x) \u2212 y| \u2265 p\n|h(x) \u2212 y| < p\n\n.\n\n(cid:26)1\n\n0\n\nObserve that l(h, z) = 0 whenever the agent wins a point and l(h, z) = 1 otherwise. If we set\nLD(h) = Ez\u223cD l(h, z), Equation (1) can be reformulated as \u03c0D(h) \u2265 suph(cid:48)\u2208H \u03c0D(h(cid:48)) \u2212 \u0001. Our\ngoal is to \ufb01nd suf\ufb01cient conditions for H to be PAC-learnable w.r.t Z and l.\nGiven H, let GH = {gh : h \u2208 H} such that\n\n\u2200h \u2208 H,\u2200z \u2208 Z : gh(z) = 1 \u2212 l(h, z) =\n\n(cid:26)1\n\n0\n\n|h(x) \u2212 y| < p\n|h(x) \u2212 y| \u2265 p\n\n.\n\nNote that GH is a class of functions from Z to {0, 1}. Suf\ufb01cient learnability conditions can now be\nstated.\nLemma 1. Let H be a class of functions from X to Y with VCdim(GH) = d < \u221e. Then there is a\nconstant C, such that for every \u0001, \u03b4 \u2208 (0, 1) and every distribution D over Z = X \u00d7 Y \u00d7 R, if we\nsample a sequence of examples S of size m \u2265 C \u00b7 d+log 1\ni.i.d. from D and pick an empirical best\nresponse h \u2208 H w.r.t. S, then with probability of at least 1 \u2212 \u03b4 it holds that\n\n\u00012\n\n\u03b4\n\n\u03c0D(h) \u2265 sup\nh(cid:48)\u2208H\n\n3 Best linear response\n\n\u03c0D(h(cid:48)) \u2212 \u0001.\n\nWe assume throughout this section that the agent uses a linear response. In what follows, we \ufb01rst\nshow that H is PAC-learnable with respect to Z and the payoff function. Afterwards, we devise an\nempirical best response algorithm with respect to a sequence of examples. Hence, according to the\nprevious section, this empirical payoff maximization algorithm outputs, with high probability, an\napproximate best response with respect to D. The proofs of all theorems and the supporting lemmas\nare in the appendix.\nFor ease of presentation, we re-denote the dimension of the instances domain to be n \u2212 1, i.e.\nX \u2282 Rn\u22121. Every h \u2208 Rn de\ufb01nes a linear predictor of a point x \u2208 Rn\u22121 via dot product, namely\n\n5\n\n\fm\n\ni=1\n\n1\n\n(cid:80)m\n\nh \u00b7 (xi, 1). Thus, Rn is referred to as the strategy space H, where axis i represents the i\u2019th entry in h,\n1 \u2264 i \u2264 n + 1. We study the case where n is \ufb01xed, although the complementary case is discussed in\nthe end of the section.\ni=1 is de-\nRecall that the empirical payoff of the agent w.r.t to a sequence of examples S = (xi, yi, pi)m\n\ufb01ned as \u03c0S (h) = 1\n|h\u00b7(xi,1)\u2212yi|<pi, and the best response w.r.t. to S is arg maxh\u2208H \u03c0S (h).\nObserve that there is a mapping MH\nS : H \u2192 {0, 1}m from any h \u2208 H to a vector v \u2208 {0, 1}m\nsuch that entry i in v equals one if h gains the i\u2019th point, and zero otherwise. Put differently,\nS (h) = v = (v1, . . . vm) such that:\nMH\n\n\u2200i \u2208 [m] : vi = 1 \u21d4 |h \u00b7 (xi, 1) \u2212 yi| < pi.\nHence, the target set of MH\nis GH(S), which is the restriction of GH to S. The size of GH(S) is\nS\nessentially the effective size of H, since any two strategies which are mapped to the same vector will\ngain the same points, and thus are equivalent. The following theorem puts a bound on the size of\ni=0 2i(cid:0)m\n(cid:80)n\nGH(S).\nTheorem 1. Let H be the hypothesis class of all linear functions in Rn\u22121. For any sequence of\nexamples S of size m, GH(S) is polynomial in m. Speci\ufb01cally, |GH(S)| \u2264\nThe VC-dimension of GH can be bounded using the Sauer - Shelah lemma [10]:\nLemma 2. It holds that VCdim(GH) \u2264 max{(cid:98)2n \u00b7 log(n)(cid:99), 20}.\nWe now devise an empirical payoff maximizing algorithm. Our approach is to \ufb01rst explicitly\ncharacterize the vectors in GH(S), and afterwards to pick a strategy from\n\n(cid:1).\n\ni\n\n(cid:27)\n\nFor each vector v, one can formulate a linear program which outputs a strategy in {h : MH\nS (h) = v}\nin case this set is not empty, or outputs none in case it is. Naively, 2m such feasibility problems can\nbe solved, although this is very inef\ufb01cient. Instead, we will recursively construct the set of feasible\nvectors. The Partial Vector Feasibility problem aids in recursively partitioning the hypothesis space.\nNote that it is solvable in time poly(n, m) using Linear Programming.\n\n(cid:26)\n\nh :\n\n(cid:13)(cid:13)MH\n\nS (h)\n\n(cid:13)(cid:13)1 = max\n\nv\u2208GH(S)(cid:107)v(cid:107)1\n\n.\n\nProblem: PARTIAL VECTOR FEASIBILITY (PVF)\nInput: a sequence of examples S = (xi, yi, pi)m\nOutput: a point h \u2208 Rn satisfying\n\n1. If vi = 1 then |h \u00b7 (xi, 1) \u2212 yi| < pi.\n2. If vi = a then h \u00b7 (xi, 1) \u2212 yi > pi.\n3. If vi = b then h \u00b7 (xi, 1) \u2212 yi < \u2212pi.\n\ni=1, and a vector v \u2208 {1, 0, a, b}m\n\nif such exists, and \u03c6 otherwise.\nThe following algorithm partitions Rn according to GH(S), where in each iteration it \"discovers\" one\nmore point in the sequence S.\nAlgorithm: EMPIRICAL PAYOFF MAXIMIZATION (EPM)\n\n// v = (v1, v2, . . . , vm)\n\ni=1\n\nInput: S = (xi, yi, pi)m\nOutput: Empirical payoff maximizer w.r.t. S\n1 v \u2190 {0}m\n2 R0 \u2190 {v}\n3 for i = 1 to m do\nRi \u2190 \u2205\n4\nfor v \u2208 Ri\u22121 do\n5\n6\n7\n8\n\nif PVF (S, (v\u2212i, \u03b1)) (cid:54)= \u03c6 then\n\nfor \u03b1 \u2208 {1, a, b} do\n\nadd (v\u2212i, \u03b1) to Ri\n9 return v\u2217 \u2208 arg maxv\u2208Rm (cid:107)v(cid:107)1\nTheorem 2. When running EPM on a sequence of examples S, it \ufb01nds an empirical best response in\npoly(|S|) time.\n\n// (v\u2212i, \u03b1) = (v1, . . . vi\u22121, \u03b1, vi+1, . . . , vm)\n\n6\n\n\fFigure 2: An example of simple linear regression with linear strategies. On the left we have\na sample sequence of size 3, along with the strategy \u00afh = (\u00afa, \u00afb) of the opponent (the solid line)\nand a best response strategy of the agent (the dashed line). On the right the hypothesis space is\npresented, where each pair (a, b) represents a possible strategy, and each bounded set Ri is de\ufb01ned by\n\n(cid:9), i.e. the set of hypotheses which give xi better prediction\n\nRi =(cid:8)(a, b) \u2208 R2 : |a \u00b7 xi + b \u2212 yi| < pi\n\nthan \u00afh. Notice that (\u00afa, \u00afb) relies on the boundaries of all Ri, 1 \u2264 i \u2264 3. In addition, since (a\u2217, b\u2217) is\ninside R1 \u2229 R2 \u2229 R3, the strategy h\u2217 = (a\u2217, b\u2217), i.e. the line y = a\u2217 \u00b7 x + b\u2217, predicts all the points\nbetter than the opponent. Observe that by taking any convex combination of h\u2217, \u00afh, the agent not only\nperserves her empirical payoff but also improves her MSE score.\n\n(cid:1)\nWhen we combine Theorem 2 with Lemmas 2 and 1, we get:\nCorollary 1. Given \u0001, \u03b4 \u2208 (0, 1), if we run EPM on m \u2265 C\n\u00012 \u00b7\nexamples sampled i.i.d. from D (for a constant C), then it outputs h\u2217 such that with probability at\nleast 1 \u2212 \u03b4 satis\ufb01es\n\n(cid:0)max{(cid:98)2n \u00b7 log(n)(cid:99), 20} + log 1\n\n\u03b4\n\n\u03c0D(h\u2217) \u2265 sup\nh(cid:48)\u2208H\n\n\u03c0D(h(cid:48)) \u2212 \u0001.\n\nA desirable achievement would be if the best response prediction algorithm would also keep the loss\nsmall in the original (e.g. MSE) measure. We now show that in some cases the agent can, by slightly\nmodifying the output of EPM, \ufb01nd a strategy that is not only an approximate best response, but is\nalso robust with respect to additive functions of discrepancies. See Figure 2 for illustration.\nLemma 3. Assume the opponent uses a linear predictor \u00afh, and denote by h\u2217 the strategy output by\nEPM. Then, h\u2217 can be ef\ufb01ciently modi\ufb01ed to a strategy which is not only an empirical best response,\nbut also performs arbitrarily close to \u00afh w.r.t. to any additive function of the discrepancies.\n\nFinaly, we discuss the case where the dimension of the instances domain is a part of the input. It is\nknown that learning the best halfspace is NP-hard in binary classi\ufb01cation (w.r.t. to a given sequence\nof points), when the dimension of the data is not \ufb01xed (see e.g. [1]). We show that the empirical best\n(linear) response problem is of the same \ufb02avor.\nLemma 4. In case H is the set of linear functions in Rn\u22121 and n is not \ufb01xed, the empirical best\nresponse problem is NP-hard.\n\n4 Experimental results\n\nWe note that when n is large, the proposed method for \ufb01nding an empirical best response may not be\nsuitable. Nevertheless, if the agent is interested in \ufb01nding a \"good\" response to her opponents, she\nshould come up with something. With slight modi\ufb01cations, the linear best response problem can be\nformulated as a mixed integer linear program (MILP).1 Hence, the agent can exploit sophisticated\nsolvers and use clever heuristics. Further, one implication of Lemma 1 is that the true payoffs\n\n1See the appendix for the mixed integer linear programming formulation.\n\n7\n\n123xyy=\u00afa\u00b7x+\u00afby=a\u2217\u00b7x+b\u2217R1R2R3(a\u2217,b\u2217)(\u00afa,\u00afb)ab\fTable 1: Experiments on Boston Housing dataset\n\nThe opponent\u2019s strategy\n\nLeast square errors (LSE)\n\nLeast absolute errors (LAE)\n\nScenario Train payoff Test payoff\nTRAIN\nALL\nTRAIN\nALL\n\n0.699\n0.711\n0.621\n0.625\n\n0.641\n0.645\n0.570\n0.528\n\nResults obtained on the Boston Housing dataset. Each cell in the table represents the average payoff\nof the agent over 1000 simulations (splits into 80% train and 20% test). The \"train payoff\" is the\nproportion of points in the training set on which the agent is more accurate, and the \"test payoff\"\npayoff is the equivalent proportion with respect to the test (unseen) data.\n\nuniformly converge, and hence any empirical payoff obtained by the MILP is close to its real payoff\nwith high probability.\nIn this section, we show the extent to which classical linear regression algorithms can be beaten\nusing the Boston housing dataset [5], a built-in dataset in the leading data science packages (e.g.\nscikit-learn in Python and MASS in R). The Boston housing dataset contains 506 instances, where\neach instance has 13 continuous attributes and one binary attribute. The label is the median value of\nowner-occupied homes, and among the attributes are the per capita crime rate, the average number of\nrooms per dwelling, the pupil-teacher ratio by town and more. The R-squared measure for minimizing\nthe square error in the Boston housing dataset is 0.74, indicating that the use of linear regression is\nreasonable.\nAs possible strategies of the opponent, we analyzed the linear least squares estimators (LSE) and\nlinear least absolute estimators (LAE). The dataset was split into training (80%) and test (20%) sets,\nand two scenarios were considered:\n\nScenario TRAIN - the opponent\u2019s model is learned from the training set only.\nScenario ALL\n\n- the opponent\u2019s model is learned from both the training and the test sets.\n\nIn both scenarios the agent had access to the training set only, along with the opponent\u2019s discrepancy\nfor each point in the training set. Obviously, achieving payoff of more than 0.5 (that is, more than\n50% of the points) in the ALL scenario is a real challenge, since the opponent has seen the test set in\nher learning process. We ran 1000 simulations, where each simulation is a random split of the dataset.\nWe employed the MILP formulation, and used Gurobi software [4] in order to \ufb01nd a response, where\nthe running time of the solver was limited to one minute.2\nOur \ufb01ndings are reported in Table 1. Notice that against both opponent strategies, and even in\ncase where the opponent had seen the test set, the agent still gets more than 50% of the points. In\nboth scenarios, LAE guarantees the opponent more than LSE. This is because absolute error is less\nsensitive to large deviations. We also noticed that when the opponent learns from the whole dataset,\nthe empirical payoff of the agent is greater. Indeed, the latter is reasonable as in the ALL scenario the\nagent\u2019s strategy \ufb01ts the training set while the opponent strategy does not.\nBeyond the main analysis, we examined the success (or lack thereof) of the agent with respect to\nthe additive loss function optimized by the opponent (corresponding to the MSE for LSE, and the\nMAE (mean absolute error) for LAE), hereby referred to as the \"classical loss\". Recall that Lemma 3\nguarantees that the agent\u2019s classical loss can be arbitrarily close to that of the opponent when she\nplays a best response; however, the response we consider in this section (using the MILP) does not\nnecessarily converge to a best response. Therefore, we \ufb01nd it interesting to consider the classical loss\nas well, thereby presenting the complementary view.\nWe report in Table 2 the average ratio between the agent\u2019s classical loss and that of the opponent\nunder the TRAIN scenario with respect to the training and test sets. Notice that the agent suffers from\nless than a 0.7% increase with respect to the classical loss optimized by the opponent. In particular,\n\n2Code\n\nreproducing\nBest-Response-Regression\n\nfor\n\nthe\n\nexperiments\n\nis\n\navailable\n\nat\n\nhttps://github.com/omerbp/\n\n8\n\n\fTable 2: Ratio of the classical loss\n\nThe opponent\u2019s strategy\nLSE\n1.007\n0.999\n\nLAE\n1.005\n1.002\n\nTraining set\nTest set\n\nRatio of the agent\u2019s loss and the opponent\u2019s loss, where the loss function corresponds to the original\noptimization function of the opponent, under scenario TRAIN. For example, the upper leftmost cell\nrepresents the agent\u2019s MSE divided by the opponents MSE on the training set, where the opponent\nuses LSE. Similarly, the lower rightmost cell represents the agent\u2019s MAE (mean absolute error)\ndivided by the opponents MAE on the test data, when the opponent uses LAE.\n\nthe MSE of the agent (when she responds to LSE) on the test set is less than that of the opponent.\nThe same phenomenon, albeit on a smaller scale, occurs against LAE: the training set ratio is greater\nthan the test set ratio.\nTo conclude, the agent is not only able to obtain the majority of the points (and in some cases, up to\n70%), but also to keep the classical loss optimized by her opponent within less than 0.2% from the\noptimum on the test set.\n\n5 Discussion\n\n(cid:0)n log n + log 1\n\n(cid:1)(cid:1)\n\n\u03b4\n\nThis work introduces a game theoretic view of a machine learning task. After \ufb01nding suf\ufb01cient\nconditions for learning to occur, we analyzed the induced learning problem, when the agent is\nrestricted to a linear response. We showed that a best response with respect to a sequence of examples\ncan be computed in polynomial time in the number of examples, as long as the instance domain has a\nconstant dimension. Further, we showed an algorithm that for any \u0001, \u03b4 computes an \u0001-best response\n\nwith a probability of at least 1 \u2212 \u03b4, when it is given a sequence of poly(cid:0) 1\n\nexamples drawn i.i.d.\nAs the reader may notice, our analysis holds as long as the hypothesis is linear in its parameters,\nand therefore is much more general than linear regression. Interestingly, this is a novel type of\noptimization problem and so rich hypothesis, which are somewhat unnatural in the traditional task of\nregression, might be successfully employed in the proposed setting.\nFrom an empirical standpoint, the gap between the empirical payoff and the true payoff calls for\napplying regularization methods for the best response problem and encourages further algorithmic\nresearch. Exploring whether or not a response in the form of hyperplanes can be effective against a\nmore complex strategy employed by the opponent will be intriguing. For instance, showing that a\ndeep learner is beatable in this setting will be remarkable.\nThe main direction to follow is the analysis of the competitive environment introduced in the beginning\nof Section 2 as a simultaneous game: is there an equilibrium strategy? Namely, is there a linear\npredictor which, when used by both the agent and the opponent, is a best response to one another?\n\n\u00012\n\nAcknowledgments\n\nWe thank Gili Baumer and Argyris Deligkas for helpful discussions, and anonymous reviewers for\ntheir useful suggestions. This project has received funding from the European Research Council (ERC)\nunder the European Union\u2019s Horizon 2020 research and innovation programme (grant agreement n\u25e6\n740435).\n\nReferences\n[1] E. Amaldi and V. Kann. The complexity and approximability of \ufb01nding maximum feasible\n\nsubsystems of linear relations. Theoretical computer science, 147(1-2):181\u2013210, 1995.\n\n[2] R. Cole and T. Roughgarden. The sample complexity of revenue maximization. In Proceedings\n\nof the 46th Annual ACM Symposium on Theory of Computing, pages 243\u2013252. ACM, 2014.\n\n9\n\n\f[3] O. Dekel, F. Fischer, and A. D. Procaccia. Incentive compatible regression learning. Journal of\n\nComputer and System Sciences, 76(8):759\u2013777, 2010.\n\n[4] I. Gurobi Optimization. Gurobi optimizer reference manual, 2016.\n\n[5] D. Harrison and D. L. Rubinfeld. Hedonic housing prices and the demand for clean air. Journal\n\nof environmental economics and management, 5(1):81\u2013102, 1978.\n\n[6] N. Immorlica, A. T. Kalai, B. Lucier, A. Moitra, A. Postlewaite, and M. Tennenholtz. Dueling\nalgorithms. In Proceedings of the forty-third annual ACM symposium on Theory of computing,\npages 215\u2013224. ACM, 2011.\n\n[7] R. Meir, A. D. Procaccia, and J. S. Rosenschein. Algorithms for strategyproof classi\ufb01cation.\n\nArti\ufb01cial Intelligence, 186:123\u2013156, 2012.\n\n[8] N. Nisan and A. Ronen. Algorithmic mechanism design. In Proceedings of the thirty-\ufb01rst\n\nannual ACM symposium on Theory of computing, pages 129\u2013140. ACM, 1999.\n\n[9] D. Pechyony and V. Vapnik. On the theory of learnining with privileged information. In\n\nAdvances in neural information processing systems, pages 1894\u20131902, 2010.\n\n[10] N. Sauer. On the density of families of sets. Journal of Combinatorial Theory, Series A, 13(1):\n\n145\u2013147, 1972.\n\n[11] S. Shalev-Shwartz and S. Ben-David. Understanding machine learning: From theory to\n\nalgorithms. Cambridge University Press, 2014.\n\n[12] L. G. Valiant. A theory of the learnable. Communications of the ACM, 27(11):1134\u20131142,\n\n1984.\n\n[13] V. Vapnik and A. Chervonenkis. On the uniform convergence of relative frequencies of events\n\nto their probabilities. Theory of Probability and its Applications, 16(2):264, 1971.\n\n[14] V. Vapnik and A. Vashist. A new learning paradigm: Learning using privileged information.\n\nNeural networks, 22(5):544\u2013557, 2009.\n\n[15] V. Vapnik, A. Vashist, and N. Pavlovitch. Learning using hidden information: Master class\nlearning. NATO Science for Peace and Security Series, D: Information and Communication\nSecurity, 19:3\u201314, 2008.\n\n10\n\n\f", "award": [], "sourceid": 963, "authors": [{"given_name": "Omer", "family_name": "Ben-Porat", "institution": "Technion \u2013 Israel Institute of Technology"}, {"given_name": "Moshe", "family_name": "Tennenholtz", "institution": "Technion--Israel Institute of Technology"}]}