{"title": "On the Adaptive Properties of Decision Trees", "book": "Advances in Neural Information Processing Systems", "page_first": 1225, "page_last": 1232, "abstract": null, "full_text": "     On the Adaptive Properties of Decision Trees\n\n\n\n              Clayton Scott                              Robert Nowak\n          Statistics Department              Electrical and Computer Engineering\n             Rice University                        University of Wisconsin\n           Houston, TX 77005                          Madison, WI 53706\n             cscott@rice.edu                         nowak@engr.wisc.edu\n\n\n\n                                          Abstract\n\n\n         Decision trees are surprisingly adaptive in three important respects: They\n         automatically (1) adapt to favorable conditions near the Bayes decision\n         boundary; (2) focus on data distributed on lower dimensional manifolds;\n         (3) reject irrelevant features. In this paper we examine a decision tree\n         based on dyadic splits that adapts to each of these conditions to achieve\n         minimax optimal rates of convergence. The proposed classifier is the\n         first known to achieve these optimal rates while being practical and im-\n         plementable.\n\n\n\n1    Introduction\n\nThis paper presents three adaptivity properties of decision trees that lead to faster rates of\nconvergence for a broad range of pattern classification problems. These properties are:\n\nNoise Adaptivity: Decision trees can automatically adapt to the (unknown) regularity of\n         the excess risk function in the neighborhood of the Bayes decision boundary. The\n         regularity is quantified by a condition similar to Tsybakov's noise condition [1].\n\nManifold Focus: When the distribution of features happens to have support on a lower di-\n         mensional manifold, decision trees can automatically detect and adapt their struc-\n         ture to the manifold. Thus decision trees learn the \"effective\" data dimension.\n\nFeature Rejection: If certain features are irrelevant (i.e., independent of the class labels),\n         then decision trees can automatically ignore these features. Thus decision trees\n         learn the \"relevant\" data dimension.\n\nEach of the above properties can be formalized and translated into a class of distributions\nwith known minimax rates of convergence. Adaptivity is a highly desirable quality of\nclassifiers since in practice the precise characteristics of the distribution are unknown.\n\nWe show that dyadic decision trees achieve the (minimax) optimal rate (to within a log\nfactor) without needing to know the specific parameters defining the class. Such trees\nare constructed by minimizing a complexity penalized empirical risk over an appropriate\nfamily of dyadic partitions. The complexity term is derived from a new generalization error\nbound for trees, inspired by [2]. This bound in turn leads to an oracle inequality from which\nthe optimal rates are derived. Full proofs of all results are given in [11].\n\n\f\nThe restriction to dyadic splits is necessary to achieve a computationally tractable classifier.\nOur classifiers have computational complexity nearly linear in the training sample size.\nThe same rates may be achieved by more general tree classifiers, but these require searches\nover prohibitively large families of partitions. Dyadic decision trees are thus preferred\nbecause they are simultaneously implementable, analyzable, and sufficiently flexible to\nachieve optimal rates.\n\n\n1.1    Notation\n\nLet Z be a random variable taking values in a set Z, and let Zn = {Z1, . . . , Zn} be iid\nrealizations of Z. Let PZ be the probability measure for Z, and let Pn be the empirical\nestimate of PZ based on Zn: Pn(B) = (1/n)                n    I\n                                                         i=1 {ZiB}, B  Z , where I denotes\nthe indicator function. In classification we take Z = X  Y, where X is the collection\nof feature vectors and Y is a finite set of class labels. Assume X = [0, 1]d, d  2, and\nY = {0, 1}. A classifier is a measurable function f : [0, 1]d  {0, 1}. Each classifier f\ninduces a set Bf = {(x, y)  Z | f (x) = y}. Define the probability of error and empirical\nerror (risk) of f by R(f ) = PZ(Bf ) and Rn(f ) = Pn(Bf ), respectively. The Bayes\nclassifier f  achieves minimum probability of error and is given by f (x) = I{(x)>1/2},\nwhere (x) = PY |X (1 |x) is the posterior probability that the correct label is 1. The\nBayes error is R(f ) and denoted R. The Bayes decision boundary, denoted G, is the\ntopological boundary of the Bayes decision set G = {x | f (x) = 1}.\n\n\n1.2    Rates of Convergence in Classification\n\n\nIn this paper we study the rate at which EZn{R(fn)} - R goes to zero as n  ,\nwhere fn is a classification learning rule, i.e., a rule for constructing a classifier from\nZn. Yang [3] shows that for (x) in certain smoothness classes minimax optimal rates\nare achieved by appropriate plug-in density estimates. Tsybakov and collaborators replace\nglobal restrictions on  by restrictions on  near G. Faster rates are then possible, al-\nthough existing optimal classifiers typically rely on -nets or otherwise non-implementable\nmethods. [1, 4, 5]. Other authors have derived rates of convergence for existing practical\nclassifiers, but these rates are suboptimal in the minimax sense considered here [68]. Our\ncontribution is to demonstrate practical classifiers that adaptively attain minimax optimal\nrates for some of Tsybakov's and other classes.\n\n\n2      Dyadic Decision Trees\n\nA dyadic decision tree (DDT) is a decision tree that divides the input space by means of\naxis-orthogonal dyadic splits. More precisely, a dyadic decision tree T is specified by\nassigning an integer s(v)  {1, . . . , d} to each internal node v of T (corresponding to the\ncoordinate/attribute that is split at that node), and a binary label 0 or 1 to each leaf node.\n\nThe nodes of DDTs correspond to hyperrectangles (cells) in [0, 1]d (see Figure 1). Given a\nhyperrectangle A =        d      [a\n                          r=1     r , br ], let As,1 and As,2 denote the hyperrectangles formed by\nsplitting A at its midpoint along coordinate s. Specifically, define As,1 = {x  A | xs \n(as + bs)/2} and As,2 = A\\As,1. Each node of a DDT is associated with a cell according\nto the following rules: (1) The root node is associated with [0, 1]d; (2) If v is an internal\nnode associated to the cell A, then the children of v are associated to As(v),1 and As(v),2.\n\nLet (T ) = {A1, . . . , Ak} denote the partition induced by T . Let j(A) denote the depth\nof A and note that (A) = 2-j(A) where  is the Lebesgue measure on Rd. Define T to\nbe the collection of all DDTs and A to be the collection of all cells corresponding to nodes\n\n\f\nFigure 1: A dyadic decision tree (right) with the associated recursive dyadic partition (left)\nwhen d = 2. Each internal node of the tree is labeled with an integer from 1 to d indicating\nthe coordinate being split at that node. The leaf nodes are decorated with class labels.\n\n\n\nof trees in T .\n\nLet M be a dyadic integer, that is, M = 2L for some nonnegative integer L. Define TM to\nbe the collection of all DDTs such that no terminal cell has a sidelength smaller than 2-L.\nIn other words, no coordinate is split more than L times when traversing a path from the\nroot to a leaf.\n\nWe will consider classifiers of the form\n\n                                     Tn = arg min Rn(T ) + n(T )                                (1)\n                                           T TM\n\n\nwhere n is a \"penalty\" or regularization term specified below. An algorithm of Blan-\nchard et al. [9] may be used to compute Tn in O(ndLd log(ndLd)) operations. For all of\nour theorems on rates of convergence below we have L = O(log n), in which case the\ncomputational cost is O(nd(log n)d+1).\n\n\n3     Generalization Error Bounds for Trees\n\n\nIn this section we state a uniform error bound and an oracle inequality for DDTs. These two\nresults are extensions of our previous work on DDTs [10]. The bounding techniques are\nquite general and can be extended to larger (even uncountable) families of trees using VC\ntheory, but for the sake of simplicity we confine the discussion to DDTs. Complete proofs\nmay be found in [11]. Before stating these results, some additional notation is necessary.\n\nLet A  A, and define A = (2 + log2 d)j(A). A represents the number of bits needed\nto uniquely encode A and will be used to measure the complexity of a DDT having A as a\nleaf cell. These \"codelengths\" satisfy a Kraft inequality                 2- A  1.\n                                                                  AA\n\nFor a cell A  [0, 1]d, define pA = PX (A) and ^\n                                                      pA = (1/n)         n    I\n                                                                         i=1 {XiA}. Further define\n^\np = 4 max(^\n                   p                                       = 4 max(p\n A                      A, ( A log 2 + log n)/n) and pA                 A, ( A log 2 + log n)/(2n)).\nIt can be shown that with high probability, pA  ^\n                                                             pA and ^pA  pA uniformly over all\nA  A [11]. The mutual boundedness of pA and ^\n                                                             pA is a key to making our proposed\nclassifier both computable on the one hand and analyzable on the other.\n\n\f\nDefine the data-dependent penalty\n\n                                                         A log 2 + log(2n)\n                       n(T ) =               2^\n                                              p                               .                     (2)\n                                                    A           n\n                                   A(T )\n\nOur first main result is the following uniform error bound.\n\nTheorem 1. With probability at least 1 - 2/n,\n\n                            R(T )  Rn(T ) + n(T ) for all T  T .                                 (3)\n\n\nTraditional error bounds for trees involve a penalty proportional to               |T | log n/n, where\n|T | denotes the number of leaves in T (see [12] or the \"naive\" bound in [2]). The penalty\nin (2) assigns a different weight to each leaf of the tree depending on both the depth of the\nleaf and the fraction of data reaching the leaf. Indeed, for very deep leaves, little data will\nreach those nodes, and such leaves will contribute very little to the overall penalty. For\nexample, we may bound ^\n                             pA by pA with high probability, and if X has a bounded density,\nthen pA decays like max{2-j, log n/n}, where j is the depth of A. Thus, as j increases,\n A grows additively with j, but ^\n                                    pA decays at a multiplicative rate. The upshot is that the\npenalty n(T ) favors unbalanced trees. Intuitively, if two trees have the same size and\nempirical error, minimizing the penalized empirical risk with this new penalty will select\nthe tree that is more unbalanced, whereas a traditional penalty based only on tree size would\nnot distinguish the two. This has advantages for classification because unbalanced trees are\nwhat we expect when approximating a lower dimensional decision boundary.\n\nThe derivation of (2) comes from applying standard concentration inequalities for sums\nof Bernoulli trials (most notably the relative Chernoff bound) in a spatially decomposed\nmanner. Spatial decomposition allows the introduction of local probabilities pA to offset\nthe complexity of each leaf node A. Our analysis is inspired by the work of Mansour and\nMcAllester [2].\n\nThe uniform error bound of Theorem 1 can be converted (using standard techniques) into\nan oracle inequality that is the key to deriving rates of convergence for DDTs.\n\nTheorem 2. Let Tn be as in (1) with n as in (2). Define\n\n                       ~                                 A log 2 + log(2n)\n                       n(T ) =               8p                              .\n                                                    A           n\n                                   A(T )\n\nThen\n                                                                                     1\n              EZn{R(Tn)} - R  min R(T ) - R + 2 ~\n                                                                 n(T ) + O               .         (4)\n                                      T T                                           n\n\nNote that with high probability, pA is an upper bound on ^\n                                                                     pA, and therefore ~\n                                                                                               n upper\nbounds n. The use of ~\n                        n instead of n in the oracle bound facilitates rate of convergence\nanalysis. The oracle inequality essentially says that Tn performs nearly as well as the\nDDT chosen by an oracle to minimize R(T ) - R. The right hand side of (4) bears the\ninterpretation of a decomposition into approximation error (R(T ) - R) and estimation\nerror ~\n        n(T ).\n\n\n4    Rates of Convergence\n\nThe classes of distributions we study are motivated by the work of Mammen and Tsybakov\n[4] and Tsybakov [1] which we now review. The classes are indexed by the smoothness\n\n\f\n of the Bayes decision boundary G and a parameter  that quantifies how \"noisy\" the\ndistribution is near G. We write an           bn when an = O(bn) and an            bn if both\nan     bn and bn     an.\n\nLet  > 0, and take r =             - 1 to be the largest integer not exceeding . Suppose\nb : [0, 1]d-1  [0, 1] is r times differentiable, and let pb,s denote the Taylor polynomial of\nb of order r at the point s. For a constant c1 > 0, define (, c1), the class of functions\nwith Holder regularity , to be the collection of all b such that\n\n                    |b(s ) - pb,s(s )|  c1|s - s | for all s, s  [0, 1]d-1.\n\nUsing Tsybakov's terminology, the Bayes decision set G is a boundary fragment of\nsmoothness  if G = epi(b) for some b  (, c1). Here epi(b) = {(s, t)  [0, 1]d :\nb(s)  t} is the epigraph of b. In other words, for a boundary fragment, the last coordinate\nof G is a Holder- function of the first d - 1 coordinates.\n\nTsybakov also introduces a condition that characterizes the level of \"noise\" near G in\nterms of a noise exponent , 1    . Let (f1, f2) = {x  [0, 1]d : f1(x) = f2(x)}.\nLet c2 > 0. A distribution satisfies Tsybakov's noise condition with noise exponent  and\nconstant c2 if\n                       PX ((f, f ))  c2(R(f ) - R)1/        for all f .                (5)\n\nThe case  = 1 is the \"low noise\" case and corresponds to a jump of (x) at the Bayes\ndecision boundary. The case  =  is the high noise case and imposes no constraint on\nthe distribution (provided c2  1). See [6] for further discussion.\n\nDefine the class F = F (, ) = F (, , c0, c1, c2) to be the collection of distributions of\nZ = (X, Y ) such that\n\n      0A For all measurable A  [0, 1]d, PX (A)  c0(A)\n\n      1A G is a boundary fragment defined by b  (, c1).\n\n      2A The margin condition is satisfied with noise exponent  and constant c2.\n\nIntroducing the parameter  = (d - 1)/, Tsybakov [1] proved the lower bound\n\n                       inf sup EZn{R(fn)} - R            n-/(2+-1).                     (6)\n                        fn    F\n\nThe inf is over all rules for constructing classifiers from training data. Theoretical rules that\nachieve this lower bound are studied by [1, 4, 5, 13]. Unfortunately, none of these works\nprovide computationally efficient algorithms for implementing the proposed discrimination\nrules, and it is unlikely that practical algorithms exist for these rules.\n\nIt is important to note that the lower bound in (6) is tight only when  < 1. To see this,\nfix  > 1. From the definition of F (, ) we have F (, 1)  F (, ) for any  > 1. As\n  , the right-hand side of (6) decreases. Therefore, the minimax rate for F(, ) can\nbe no faster than n-1/(1+), which is the lower bound for F (, 1).\n\nIn light of the above, Tsybakov's noise condition does not improve the learning situation\nwhen  > 1. To achieve rates faster than n-1/(1+) when  > 1, clearly an alternate\nassumption must be made. If the right-hand side of (6) is any indication, then the distribu-\ntions responsible for slower rates are those with small . Thus, it would seem that we need\na noise assumption that excludes those \"low noise\" distributions with small  that cause\nslower rates when  > 1.\n\nSince recursive dyadic partitions can well-approximate G with smoothness   1, we\nare in the regime of   (d - 1)/  1. As motivated above, faster rates in this situation\nrequire an assumption that excludes low noise levels. We propose such an assumption. Like\n\n\f\nTsybakov's noise condition, our assumption is also defined in terms of constants   1\nand c2 > 0. Because of limited space we are unable to fully present the modified noise\ncondition, and we simply write\n\n       2B Low noise levels are excluded as defined in [11].\n\nEffectively, 2B says that the inequality in (5) is reversed, not for all classifiers f , but only\nfor those f that are the best DDT approximations to f  for each DDT resolutions parameter\nM . Using techniques presented in [13], we show in [11] that lower bounds of the form in\n(6) are valid when 2A is replaced by 2B. According to the results in the next section, these\nlower bounds are tight to within a log factor for  > 1.\n\n\n5      Adaptive Rates for Dyadic Decision Trees\n\nAll of our rate of convergence proofs use the oracle inequality in the same basic way. The\nobjective is to find an \"oracle tree\" T   T such that both R(T ) - R and ~\n                                                                                n(T ) decay\nat the desired rate. This tree is roughly constructed as follows. First form a \"regular\"\ndyadic partition (the exact construction will depend on the specific problem) into cells\nof sidelength 1/m = 2-K , for a certain K  L. Then \"prune back\" all cells that do\nnot intersect G. Both approximation and estimation error may now be bounded using\nthe given assumptions and elementary bounding methods. For example, R(T ) - R\n(PZ((T , f))) (by 2B)         (((T , f ))) (by 0A)      m- (by 1A). This example\nreveals how the noise exponent enters the picture to affect the approximation error. See [11]\nfor complete proofs.\n\n\n5.1    Noise Adaptive Classification\n\nDyadic decision trees, selected according to the penalized empirical risk criterion discussed\nearlier, adapt to the unknown noise level to achieve faster rates as stated in Theorem 3\nbelow. For now we focus on distributions with  = 1 ( = d - 1), i.e., Lipschitz decision\nboundaries (the case  = 1 is discussed in Section 5.4), and arbitrary noise parameter .\nThe optimal rate for this class is n-/(2+d-2) [11]. We will see that DDTs can adaptively\nlearn at a rate of (log n/n)/(2+d-2).\n\nIn an effort to be more general and practical, we replace the boundary fragment condition\n1A with a less restrictive assumption. Tysbakov and van de Geer [5] assume the Bayes\ndecision set G is a boundary fragment, meaning it is known a priori that (a) one coordinate\nof G is a function of the others, (b) that coordinate is known, and (c) class 1 corresponds\nto the region above G. The following condition includes all piecewise Lipschitz decision\nboundaries, and allows G to have arbitrary orientation and G to have multiple connected\ncomponents. Let Pm denote the regular partition of [0, 1]d into hypercubes of sidelength\n1/m where m is a dyadic integer (i.e., a power of 2). A distribution of Z satisfies the\nbox-counting assumption with constant c1 > 0 if\n\n       1B For all dyadic integers m, G intersects at most c1md-1 of the md cells in Pm.\n\nCondition 1A ( = 1) implies 1B, (with a different c1) so the minimax rate under 0A, 1B,\nand 2B is no faster than n-/(2+d-2).\n\nTheorem 3. Let M         (n/ log n). Take Tn as in (1) with n as in (2). Then\n                                                                   \n                                                                 2+d-2\n                        sup EZn{R(Tn)} - R             log n              .                (7)\n                                                            n\n\nwhere the sup is over all distributions such that 0A, 1B, and 2B hold.\n\n\f\nThe complexity regularized DDT is adaptive in the sense that the noise exponent  and\nconstants c0, c1, c2 need not be known. Tn can always be constructed and in opportune\ncircumstances the rate in (7) is achieved.\n\n\n5.2      When the Data Lie on a Manifold\n\nFor certain problems it may happen that the feature vectors lie on a manifold in the ambient\nspace X . When this happens, dyadic decision trees automatically adapt to achieve faster\nrates of convergence. To recast assumptions 0A and 1B in terms of a data manifold1, we\nagain use box-counting ideas. Let c0, c1 > 0 and 1  d  d. The boundedness and\nregularity assumptions for a d dimensional manifold are given by\n\n\n        0B For all dyadic integers m and all A  Pm, PX (A)  c0m-d .\n\n        1C For all dyadic integers m, G passes through at most c1md -1 of the md hyper-\n             cubes in Pm.\n\nThe minimax rate under these assumptions is n-1/d . To see this, consider the mapping of\nfeatures X = (X1, . . . , Xd )  [0, 1]d to X = (X1, . . . , Xd , 1/2, . . . , 1/2)  [0, 1]d.\nThen X lives on a d dimensional manifold, and clearly there can be no classifier achieving\na rate faster than n-1/d uniformly over all such X, as this would lead to a classifier out-\nperforming the minimax rate for X . As the following theorem shows, DDTs can achieve\nthis rate to within a log factor.\n\nTheorem 4. Let M             (n/ log n). Take Tn as in (1) with n as in (2). Then\n\n                                                                         1\n                                                                         d\n                             sup EZn{R(Tn)} - R                log n         .                    (8)\n                                                                   n\n\nwhere the sup is over all distributions such that 0B and 1C hold.\n\n\nAgain, Tn is adaptive in that it does not require knowledge d , c0, or c1.\n\n\n5.3      Irrelevant Features\n\nThe \"relevant\" data dimension is the number of relevant features/attributes, meaning the\nnumber d < d of features of X that are not independent of Y . By an argument like that in\nthe previous section, the minimax rate under this assumption (and 0A and 1B) can be seen\nto be n-1/d . Once again, DDTs can achieve this rate to within a log factor.\n\nTheorem 5. Let M             (n/ log n). Take Tn as in (1) with n as in (2).\n\n                                                                         1\n                                                                         d\n                             sup EZn{R(Tn)} - R                log n         .                    (9)\n                                                                  n\n\nwhere the sup is over all distributions with relevant data dimension d and such that 0A\nand 1B hold.\n\nAs in the previous theorems, our learning rule is adaptive in the sense that it does not need\nto be told d or which d features are relevant.\n\n       1For simplicity, we eliminate the margin assumption here and in subsequent sections, although it\ncould be easily incorporated to yield faster adaptive rates.\n\n\f\n5.4    Adapting to Bayes Decision Boundary Smoothness\n\nOur results thus far apply to Tsybakov's class with  = 1. In [10] we show that DDTs with\npolynomial classifiers decorating the leaves can achieve faster rates for  > 1. Combined\nwith the analysis here, these rates can approach n-1 under appropriate noise assumptions.\nUnfortunately, the rates we obtain are suboptimal and the classifiers are not practical.\n\nFor   1, free DDTs adaptively attain the minimax rate (within a log factor) of\nn-/(+d-1). Due to space limitations, this discussion is deferred to [11]. Finding practi-\ncal classifiers that adapt to the optimal rate for  > 1 is a current line of research.\n\n\n6      Conclusion\n\nDyadic decision trees adapt to a variation of Tsybakov's noise condition, data manifold\ndimension and the number of relevant features to achieve minimax optimal rates of con-\nvergence (to within a log factor). DDTs are constructed by a computationally efficient\npenalized empirical risk minimization procedure based on a novel, spatially adaptive, data-\ndependent penalty. Although we consider each condition separately so as to simplify the\ndiscussion, the conditions can be combined to yield a rate of (log n/n)/(2+d-2) where\nd is the dimension of the manifold supporting the relevant features.\n\nReferences\n\n [1] A. B. Tsybakov, \"Optimal aggregation of classifiers in statistical learning,\" Ann. Stat., vol. 32,\n       no. 1, pp. 135166, 2004.\n\n [2] Y. Mansour and D. McAllester, \"Generalization bounds for decision trees,\" in Proceedings of\n       the Thirteenth Annual Conference on Computational Learning Theory, N. Cesa-Bianchi and\n       S. Goldman, Eds., Palo Alto, CA, 2000, pp. 6974.\n\n [3] Y. Yang, \"Minimax nonparametric classificationPart I: Rates of convergence,\" IEEE Trans.\n       Inform. Theory, vol. 45, no. 7, pp. 22712284, 1999.\n\n [4] E. Mammen and A. B. Tsybakov, \"Smooth discrimination analysis,\" Ann. Stat., vol. 27, pp.\n       18081829, 1999.\n\n [5] A. B. Tsybakov and S. A. van de Geer, \"Square root penalty: adaptation to the margin in\n       classification and in edge estimation,\" 2004, preprint.\n\n [6] P. Bartlett, M. Jordan, and J. McAuliffe, \"Convexity, classification, and risk bounds,\" Depart-\n       ment of Statistics, U.C. Berkeley, Tech. Rep. 638, 2003, to appear in Journal of the American\n       Statistical Association.\n\n [7] G. Blanchard, G. Lugosi, and N. Vayatis, \"On the rate of convergence of regularized boosting\n       classifiers,\" J. Machine Learning Research, vol. 4, pp. 861894, 2003.\n\n [8] J. C. Scovel and I. Steinwart, \"Fast rates for support vector machines,\" Los Alamos National\n       Laboratory, Tech. Rep. LA-UR 03-9117, 2004.\n\n [9] G. Blanchard, C. Schafer, and Y. Rozenholc, \"Oracle bounds and exact algorithm for dyadic\n       classification trees,\" in Learning Theory: 17th Annual Conference on Learning Theory, COLT\n       2004, J. Shawe-Taylor and Y. Singer, Eds.    Heidelberg: Springer-Verlag, 2004, pp. 378392.\n\n[10] C. Scott and R. Nowak, \"Near-minimax optimal classification with dyadic classification trees,\"\n       in Advances in Neural Information Processing Systems 16, S. Thrun, L. Saul, and B. Scholkopf,\n       Eds.    Cambridge, MA: MIT Press, 2004.\n\n[11] ----, \"Minimax optimal classification with dyadic decision trees,\" Rice University, Tech. Rep.\n       TREE0403, 2004. [Online]. Available: http://www.stat.rice.edu/cscott\n[12] A. Nobel, \"Analysis of a complexity based pruning scheme for classification trees,\" IEEE Trans.\n       Inform. Theory, vol. 48, no. 8, pp. 23622368, 2002.\n\n[13] J.-Y. Audibert, \"PAC-Bayesian statistical learning theory,\" Ph.D. dissertation, Universite Paris\n       6, June 2004.\n\n\f\n", "award": [], "sourceid": 2625, "authors": [{"given_name": "Clayton", "family_name": "Scott", "institution": null}, {"given_name": "Robert", "family_name": "Nowak", "institution": null}]}