{"title": "A Fourier Perspective on Model Robustness in Computer Vision", "book": "Advances in Neural Information Processing Systems", "page_first": 13276, "page_last": 13286, "abstract": "Achieving robustness to distributional shift is a longstanding and challenging goal of computer vision. Data augmentation is a commonly used approach for improving robustness, however robustness gains are typically not uniform across corruption types. Indeed increasing performance in the presence of random noise is often met with reduced performance on other corruptions such as contrast change. Understanding when and why these sorts of trade-offs occur is a crucial step towards mitigating them. Towards this end, we investigate recently observed trade-offs caused by Gaussian data augmentation and adversarial training. We find that both methods improve robustness to corruptions that are concentrated in the high frequency domain while reducing robustness to corruptions that are concentrated in the low frequency domain. This suggests that one way to mitigate these trade-offs via data augmentation is to use a more diverse set of augmentations.\nTowards this end we observe that AutoAugment, a recently proposed data augmentation policy optimized for clean accuracy, achieves state-of-the-art robustness on the CIFAR-10-C benchmark.", "full_text": "A Fourier Perspective on Model Robustness in\n\nComputer Vision\n\nDepartment of EECS\n\nUC Berkeley\n\nDong Yin\u2217\n\nBerkeley, CA 94720\n\ndongyin@berkeley.edu\n\nRaphael Gontijo Lopes\u2020\nGoogle Research, Brain team\nMountain View, CA 94043\niraphael@google.com\n\nJonathon Shlens\n\nGoogle Research, Brain team\nMountain View, CA 94043\n\nshlens@google.com\n\nEkin D. Cubuk\n\nGoogle Research, Brain team\nMountain View, CA 94043\n\ncubuk@google.com\n\nJustin Gilmer\n\nGoogle Research, Brain team\nMountain View, CA 94043\n\ngilmer@google.com\n\nAbstract\n\nAchieving robustness to distributional shift is a longstanding and challenging\ngoal of computer vision. Data augmentation is a commonly used approach for\nimproving robustness, however robustness gains are typically not uniform across\ncorruption types. Indeed increasing performance in the presence of random noise is\noften met with reduced performance on other corruptions such as contrast change.\nUnderstanding when and why these sorts of trade-offs occur is a crucial step\ntowards mitigating them. Towards this end, we investigate recently observed trade-\noffs caused by Gaussian data augmentation and adversarial training. We \ufb01nd that\nboth methods improve robustness to corruptions that are concentrated in the high\nfrequency domain while reducing robustness to corruptions that are concentrated in\nthe low frequency domain. This suggests that one way to mitigate these trade-offs\nvia data augmentation is to use a more diverse set of augmentations. Towards this\nend we observe that AutoAugment [6], a recently proposed data augmentation\npolicy optimized for clean accuracy, achieves state-of-the-art robustness on the\nCIFAR-10-C [17] benchmark.\n\nIntroduction\n\n1\nAlthough many deep learning computer vision models achieve remarkable performance on many\nstandard i.i.d benchmarks, these models lack the robustness of the human vision system when the train\nand test distributions differ [24]. For example, it has been observed that commonly occurring image\ncorruptions, such as random noise, contrast change, and blurring, can lead to signi\ufb01cant performance\ndegradation [8, 3]. Improving distributional robustness is an important step towards safely deploying\nmodels in complex, real-world settings.\nData augmentation is a natural and sometimes effective approach to learning robust models. Examples\nof data augmentation include adversarial training [14], applying image transformations to the training\ndata, such as \ufb02ipping, cropping, adding random noise, and even stylized image transformation [11].\nHowever, data augmentation rarely improves robustness across all corruption types. Performance\ngains on some corruptions may be met with dramatic reduction on others. As an example, in [10] it\n\n\u2217Work done while internship at Google Research, Brain team.\n\u2020Work done as a member of the Google AI Residency program g.co/airesidency.\n\n33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada.\n\n\fwas observed that Gaussian data augmentation and adversarial training improve robustness to noise\nand blurring corruptions on the CIFAR-10-C and ImageNet-C common corruption benchmarks [17],\nwhile signi\ufb01cantly degrading performance on the fog and contrast corruptions. This begs a natural\nquestion\nWhat is different about the corruptions for which augmentation strategies improve performance vs.\nthose which performance is degraded?\nUnderstanding these tensions and why they occur is an important \ufb01rst step towards designing robust\nmodels. Our operating hypothesis is that the frequency information of these different corruptions\noffers an explanation of many of these observed trade-offs. Through extensive experiments involving\nperturbations in the Fourier domain, we demonstrate that these two augmentation procedures bias\nthe model towards utilizing low frequency information in the input. This low frequency bias results\nin improved robustness to corruptions which are more high frequency in nature while degrading\nperformance on corruptions which are low frequency.\nOur analysis suggests that more diverse data augmentation procedures could be leveraged to mitigate\nthese observed trade-offs, and indeed this appears to be true. In particular we demonstrate that the\nrecently proposed AutoAugment data augmentation policy [6] achieves state-of-the-art results on\nthe CIFAR-10-C benchmark. In addition, a follow-up work has utilized AutoAugment in a way to\nachieve state-of-the-art results on ImageNet-C [1].\nSome of our observations could be of interest to research on security. For example, we observe\nperturbations in the Fourier domain which when applied to images cause model error rates to exceed\n90% on ImageNet while preserving the semantics of the image. These qualify as simple, single\nquery3 black box attacks that satisfy the content preserving threat model [13]. This observation was\nalso made in concurrent work [26].\nFinally, we extend our frequency analysis to obtain a better understanding of worst-case perturbations\nof the input. In particular adversarial perturbations of a naturally trained model are more high-\nfrequency in nature while adversarial training encourages these perturbations to become more\nconcentrated in the low frequency domain.\n2 Preliminaries\nWe denote the (cid:96)2 norm of vectors (and in general, tensors) by (cid:107) \u00b7 (cid:107). For a vector x \u2208 Rd, we denote\nits entries by x[i], i \u2208 {0, . . . , d \u2212 1}, and for a matrix X \u2208 Rd1\u00d7d2, we denote its entries by X[i, j],\ni \u2208 {0, . . . , d1 \u2212 1}, j \u2208 {0, . . . , d2 \u2212 1}. We omit the dimension of image channels, and denote\nthem by matrices X \u2208 Rd1\u00d7d2. We denote by F : Rd1\u00d7d2 \u2192 Cd1\u00d7d2 the 2D discrete Fourier\ntransform (DFT) and by F\u22121 the inverse DFT. When we visualize the Fourier spectrum, we always\nshift the low frequency components to the center of the spectrum.\nWe de\ufb01ne high pass \ufb01ltering with bandwidth B as the operation that sets all the frequency components\noutside of a centered square with width B in the Fourier spectrum with highest frequency in the\ncenter to zero, and then applies inverse DFT. The low pass \ufb01ltering operation is de\ufb01ned similarly\nwith the difference that the centered square is applied to the Fourier spectrum with low frequency\nshifted to the center.\nWe assume that the pixels take values in range [0, 1]. In all of our experiments with data augmentation\nwe always clip the pixel values to [0, 1]. We de\ufb01ne Gaussian data augmentation with parameter \u03c3\n\nas the following operation: In each iteration, we add i.i.d. Gaussian noise N (0,(cid:101)\u03c32) to every pixel\nin all the images in the training batch, where(cid:101)\u03c3 is chosen uniformly at random from [0, \u03c3]. For our\n\nexperiments on CIFAR-10, we use the Wide ResNet-28-10 architecture [27], and for our experiment\non ImageNet, we use the ResNet-50 architecture [16]. When we use Gaussin data augmentation, we\nchoose parameter \u03c3 = 0.1 for CIFAR-10 and \u03c3 = 0.4 for ImageNet. All experiments use \ufb02ip and\ncrop during training.\nFourier heat map We will investigate the sensitivity of models to high and low frequency corrup-\ntions via a perturbation analysis in the Fourier domain. Let Ui,j \u2208 Rd1\u00d7d2 be a real-valued matrix\nsuch that (cid:107)Ui,j(cid:107) = 1, and F(Ui,j) only has up to two non-zero elements located at (i, j) and the its\nsymmetric coordinate with respect to the image center; we call these matrices the 2D Fourier basis\nmatrices [4].\n\n3In contrast, methods for generating small adversarial perturbations require 1000\u2019s of queries [15].\n\n2\n\n\fMore speci\ufb01cally, we can compute (cid:101)Xi,j = X + rvUi,j, where r is chosen uniformly at random from\n\nGiven a model and a validation image X, we can generate a perturbed image with Fourier basis noise.\n{\u22121, 1}, and v > 0 is the norm of the perturbation. For multi-channel images, we perturb every\nchannel independently. We can then evaluate the models under Fourier basis noise and visualize\nhow the test error changes as a function of (i, j), and we call these results the Fourier heat map of a\nmodel. We are also interested in understanding how the outputs of the models\u2019 intermediate layers\nchange when we perturb the images using a speci\ufb01c Fourier basis, and these results are relegated to\nthe Appendix.\n3 The robustness problem\n\nFigure 1: Models can achieve high accuracy using information from the input that would be unrecognizable to\nhumans. Shown above are models trained and tested with aggressive high and low pass \ufb01ltering applied to the\ninputs. With aggressive low-pass \ufb01ltering, the model is still above 30% on ImageNet when the images appear to\nbe simple globs of color. In the case of high-pass (HP) \ufb01ltering, models can achieve above 50% accuracy using\nfeatures in the input that are nearly invisible to humans. As shown on the right hand side, the high pass \ufb01ltered\nimages needed be normalized in order to properly visualize the high frequency features (the method that we use\nto visualize the high pass \ufb01ltered images is provided in the appendix).\nHow is it possible that models achieve such high performance in the standard settings where the\ntraining and test data are i.i.d., while performing so poorly in the presence of even subtle distributional\nshift? There has been substantial prior work towards obtaining a better understanding of the robust-\nness problem. While this problem is far from being completely understood, perhaps the simplest\nexplanation is that models lack robustness to distributional shift simply because there is no reason for\nthem to be robust [20, 11, 18]. In naturally occurring data there are many correlations between the\ninput and target that models can utilize to generalize well. However, utilizing such suf\ufb01cient statistics\nwill lead to dramatic reduction in model performance should these same statistics become corrupted\nat test time.\nAs a simple example of this principle, consider Figure 8 in [19]. The authors experimented with\ntraining models on a \u201ccheating\u201d variant of MNIST, where the target label is encoded by the location\nof a single pixel. Models tested on images with this \u201ccheating\u201d pixel removed would perform poorly.\nThis is an unfortunate setting where Occam\u2019s razor can fail. The simplest explanation of the data\nmay generalize well in perfect settings where the training and test data are i.i.d., but fail to generalize\nrobustly. Although this example is arti\ufb01cial, it is clear that model brittleness is tied to latching onto\nnon-robust statistics in naturally occurring data.\nAs a more realistic example, consider the recently proposed texture hypothesis [11]. Models trained\non natural image data can obtain high classi\ufb01cation performance relying on local statistics that\nare correlated with texture. However, texture-like information can become easily distorted due to\nnaturally occurring corruptions caused by weather or digital artifacts, leading to poor robustness.\nIn the image domain, there is a plethora of correlations between the input and target. Simple statistics\nsuch as colors, local textures, shapes, even unintuitive high frequency patterns can all be leveraged\nin a way to achieve remarkable i.i.d generalization. To demonstrate, we experimented with training\n\n3\n\n\fand testing of ImageNet models when severe \ufb01ltering is performed on the input in the frequency\ndomain. While modest \ufb01ltering has been used for model compression [9], we experiment with\nextreme \ufb01ltering in order to test the limits of model generalization. The results are shown in Figure 1.\nWhen low-frequency \ufb01ltering is applied, models can achieve over 30% test accuracy even when the\nimage appears to be simple globs of color. Even more striking, models achieve 50% accuracy in\nthe presence of the severe high frequency \ufb01ltering, using high frequency features which are nearly\ninvisible to humans. In order to even visualize these high frequency features, we had normalize pixel\nstatistics to have unit variance. Given that these types features are useful for generalization, it is not\nso surprising that models leverage these non-robust statistics.\nIt seems likely that these invisible high frequency features are related to the experiments of [18], which\nshow that certain imperceptibly perturbed images contain features which are useful for generalization.\nWe discuss these connections more in Section 4.4.\n\n4 Trade-off and correlation between corruptions: a Fourier perspective\nThe previous section demonstrated that both high and low frequency features are useful for classi\ufb01ca-\ntion. A natural hypothesis is that data augmentation may bias the model towards utilizing different\nkinds of features in classi\ufb01cation. What types of features models utilize will ultimately determine the\nrobustness at test time. Here we adopt a Fourier perspective to study the trade-off and correlation\nbetween corruptions when we apply several data augmentation methods.\n\n4.1 Gaussian data augmentation and adversarial training bias models towards low\n\nfrequency information\n\nFord et al. [10] investigated the robustness of three models on CIFAR-10-C: a naturally trained\nmodel, a model trained by Gaussian data augmentation, and an adversarially trained model. It was\nobserved that Gaussian data augmentation and adversarial training improve robustness to all noise\nand many of the blurring corruptions, while degrading robustness to fog and contrast. For example\nadversarial training degrades performance on the most severe contrast corruption from 85.66% to\n55.29%. Similar results were reported on ImageNet-C.\n\nFigure 2: Left: Fourier spectrum of natural images; we estimate E[|F(X)[i, j]|] by averaging all the CIFAR-10\nvalidation images. Right: Fourier spectrum of the corruptions in CIFAR-10-C at severity 3. For each corruption,\nwe estimate E[|F(C(X) \u2212 X)[i, j]|] by averaging over all the validation images. Additive noise has relatively\nhigh concentrations in high frequencies while some corruptions such as fog and contrast are concentrated in low\nfrequencies.\n\nWe hypothesize that some of these trade-offs can be explained by the Fourier statistics of different\ncorruptions. Denote a (possibly randomized) corruption function by C : Rd1\u00d7d2 \u2192 Rd1\u00d7d2. In\nFigure 2 we visualize the Fourier statistics of natural images as well as the average delta of the\ncommon corruptions. Natural images have higher concentrations in low frequencies, thus when we\nrefer to a \u201chigh\u201d or \u201clow\u201d frequency corruption we will always use this term on a relative scale.\nGaussian noise is uniformly distributed across the Fourier frequencies and thus has much higher\nfrequency statistics relative to natural images. Many of the blurring corruptions remove or change the\nhigh frequency content of images. As a result C(X)\u2212 X will have a higher fraction of high frequency\n\n4\n\nclean imagesbrightnesscontrastdefocus blurelasticfogGaussian blurglass blurimpulse noisejpegmotion blurpixelateshot noisesnowspeckle noisezoom blur\fFigure 3: Model sensitivity to additive noise aligned with different Fourier basis vectors on CIFAR-10. We\n\ufb01x the additive noise to have (cid:96)2 norm 4 and evaluate three models: a naturally trained model, an adversarially\ntrained model, and a model trained with Gaussian data augmentation. Error rates are averaged over 1000\nrandomly sampled images from the test set. In the bottom row we show images perturbed with noise along the\ncorresponding Fourier basis vector. The naturally trained model is highly sensitive to additive noise in all but the\nlowest frequencies. Both adversarial training and Gaussian data augmentation dramatically improve robustness\nin the higher frequencies while sacri\ufb01cing the robustness of the naturally trained model in the lowest frequencies\n(i.e. in both models, blue area in the middle is smaller compared to that of the naturally trained model).\n\nenergy. For corruptions such as contrast and fog, the energy of the corruption is concentrated more\non low frequency components.\nThe observed differences in the Fourier statistics suggests an explanation for why the two augmenta-\ntion methods improve performance in additive noise but not fog and contrast \u2014 the two augmentation\nmethods encourage the model to become invariant to high frequency information while relying more\non low frequency information. We investigate this hypothesis via several perturbation analyses of\nthe three models in question. First, we test model sensitivity to perturbations along each Fourier\nbasis vector. Results on CIFAR-10 are shown in Figure 3. The difference between the three models\nis striking. The naturally trained model is highly sensitive to additive perturbations in all but the\nlowest frequencies, while Gaussian data augmentation and adversarial training both dramatically\nimprove robustness in the higher frequencies. For the models trained with data augmentation, we\nsee a subtle but distinct lack of robustness at the lowest frequencies (relative to the naturally trained\nmodel). Figure 4 shows similar results for three different models on ImageNet. Similar to CIFAR-10,\nGaussian data augmentation improves robustness to high frequency perturbations while reducing\nperformance on low frequency perturbations.\n\nFigure 4: Model sensitivity to additive noise aligned with different Fourier basis vectors on ImageNet validation\nimages. We \ufb01x the basis vectors to have (cid:96)2 norm 15.7. Error rates are averaged over the entire ImageNet\nvalidation set. We present the 63 \u00d7 63 square centered at the lowest frequency in the Fourier domain. Again, the\nnaturally trained model is highly sensitive to additive noise in all but the lowest frequencies. On the other hand,\nGaussian data augmentation improves robustness in the higher frequencies while sacri\ufb01cing the robustness to\nlow frequency perturbations. For AutoAugment, we observe that its Fourier heat map has the largest blue/yellow\narea around the center, indicating that AutoAugment is relatively robust to low to mid frequency corruptions.\n\n5\n\n\fTo test this further, we added noise with \ufb01xed (cid:96)2 norm but different frequency bandwidths centered at\nthe origin. We consider two settings, one where the origin is centered at the lowest frequency and\none where the origin is centered at the highest frequency. As shown in Figure 5, for a low frequency\ncentered bandwidth of size 3, the naturally trained model has less than half the error rate of the other\ntwo models. For high frequency bandwidth, the models trained with data augmentation dramatically\noutperform the naturally trained model.\n\nFigure 5: Robustness of models under additive noise with \ufb01xed norm and different frequency distribution. For\neach channel in each CIFAR-10 test image, we sample i.i.d Gaussian noise, apply a low/high pass \ufb01lter, and\nnormalize the \ufb01ltered noise to have (cid:96)2 norm 8, before applying to the image. We vary the bandwidth of the\nlow/high pass \ufb01lter and generate the two plots. The naturally trained model is more robust to the low frequency\nnoise with bandwidth 3, while Gaussian data augmentation and adversarial training make the model more robust\nto high frequency noise.\n\nThis is consistent with the hypothesis that the models trained with the noise augmentation are biased\ntowards low frequency information. As a \ufb01nal test, we analyzed the performance of models with a\nlow/high pass \ufb01lter applied to the input (we call the low/high pass \ufb01lters the front end of the model).\nConsistent with prior experiments we \ufb01nd that applying a low pass front-end degrades performance\non fog and contrast while improving performance on additive noise and blurring. If we instead\nfurther bias the model towards high frequency information we observe the opposite effect. Applying\na high-pass front end degrades performance on all corruptions (as well as clean test error), but\nperformance degradation is more severe on the high frequency corruptions. These experiments again\ncon\ufb01rm our hypothesis about the robustness properties of models with a high (or low) frequency bias.\nTo better quantify the relationship between frequency and robustness for various models we measure\nthe ratio of energy in the high and low frequency domain. For each corruption C, we apply high\npass \ufb01ltering with bandwidth 27 (denote this operation by H(\u00b7)) on the delta of the corruption, i.e.,\nC(X) \u2212 X. We use (cid:107)H(C(X)\u2212X)(cid:107)2\nas a metric of the fraction of high frequency energy in the\n(cid:107)C(X)\u2212X(cid:107)2\ncorruption. For each corruption, we average this quantity over all the validation images and all\n5 severities. We evaluate 6 models on CIFAR-10-C, each trained differently \u2014 natural training,\nGaussian data augmentation, adversarial training, trained with a low pass \ufb01lter front end (bandwidth\n15), trained with a high pass \ufb01lter front end (bandwidth 31), and trained with AutoAugment (see a\nmore detailed discussion on AutoAugment in Section 4.3). Results are shown in Figure 6. Models\nwith a low frequency bias perform better on the high frequency corruptions. The model trained with\na high pass \ufb01lter has a forced high frequency bias. While this model performs relatively poorly on\neven natural data, it is clear that high frequency corruptions degrade performance more than the low\nfrequency corruptions. Full results, including those on ImageNet, can be found in the appendix.\n4.2 Does low frequency data augmentation improve robustness to low frequency\n\ncorruptions?\n\nWhile Figure 6 shows a clear relationship between frequency and robustness gains of several data\naugmentation strategies, the Fourier perspective is not predictive in all situations of transfer between\ndata augmentation and robustness.\nWe experimented with applying additive noise that matches the statistics of the fog corruption in\ni,j)Ui,j\nwhere the \u03c3i,j are chosen to match the typical norm of the fog corruption on basis vector Ui,j as\n\nthe frequency domain. We de\ufb01ne \u201cfog noise\u201d to be the additive noise distribution(cid:80)\n\nN (0, \u03c32\n\ni,j\n\n6\n\n051015202530bandwidth0.00.20.40.60.81.0test acclow pass filtered noise051015202530bandwidth0.00.20.40.60.81.0test acchigh pass filtered noisenaturally trainedGaussian augmentationadversarially trained\fFigure 6: Relationship between test accuracy and fraction of high frequency energy of the CIFAR-10-C\ncorruptions. Each scatter point in the plot represents the evaluation result of a particular model on a particular\ncorruption type. The x-axis represents the fraction of high frequency energy of the corruption type, and the y-axis\nrepresents change in test accuracy compared to a naturally trained model. Overall, Gaussian data augmentation,\nadversarial training, and adding low pass \ufb01lter improve robustness to high frequency corruptions, and degrade\nrobustness to low frequency corruptions. Applying a high pass \ufb01lter front end yields a more signi\ufb01cant accuracy\ndrop on high frequency corruptions compared to low frequency corruptions. AutoAugment improves robustness\non nearly all corruptions, and achieves the best overall performance. The legend at the bottom shows the slope\n(k) and residual (r) of each \ufb01tted line.\n\nshown in Figure 2. In particular, the marginal statistics of fog noise are identical to the fog corruption\nin the Fourier domain. However, data augmentation on fog noise degrades performance on the fog\ncorruption (Table 1). This occurs despite the fact that the resulting model yields improved robustness\nto perturbations along the low frequency vectors (see the Fourier heat maps in the appendix).\n\nfog severity\n\nnaturally trained\n\nfog noise augmentation\n\n1\n\n0.9606\n0.9090\n\n2\n\n0.9484\n0.8726\n\n3\n\n0.9395\n0.8120\n\n4\n\n0.9072\n0.7175\n\n5\n\n0.7429\n0.4626\n\nTable 1: Training with fog noise hurts performance on fog corruption.\n\nWe hypothesize that the story is more complicated for low frequency corruptions because of an\nasymmetry between high and low frequency information in natural images. Given that natural images\nare concentrated more in low frequencies, a model can more easily learn to \u201cignore\u201d high frequency\ninformation rather than low frequency information. Indeed as shown in Figure 1, model performance\ndrops off far more rapidly when low frequency information is removed than high.\n4.3 More varied data augmentation offers more general robustness\nThe trade-offs between low and high frequency corruptions for Gaussian data augmentation and\nadversarial training lead to the natural question of how to achieve robustness to a more diverse\nset of corruptions. One intuitive solution is to train on a variety of data augmentation strategies.\nTowards this end, we investigated the learned augmentation policy AutoAugment [6]. AutoAugment\napplies a learned mixture of image transformations during training and achieves the state-of-the-\nart performance on CIFAR-10 and ImageNet. In all of our experiments with AutoAugment, we\nremove the brightness and constrast sub-policies as they explicitly appear in the common corruption\nbenchmarks. 4 Despite the fact that this policy was tuned speci\ufb01cally for clean test accuracy, we\nfound that it also dramatically improves robustness on CIFAR-10-C. Here, we demonstrate part of\nthe results in Table 2, and the full results can be found in the appendix. In the third plot in Figure 6,\nwe also visualize the performance of AutoAugment on CIFAR-10-C.\nMore speci\ufb01cally, on CIFAR-10-C, we compare the robustness of the naturally trained model,\nGaussian data augmentation, adversarially trained model, and AutoAugment. We observe that among\nthe four models, AutoAugment achieves the best average corruption test accuracy of 86%. Using\nthe mean corruption error (mCE) metric proposed in [17] with the naturally trained model being the\nbaseline (see a formal de\ufb01nition of mCE in the appendix), we observe that AutoAugment achieves\nthe best mCE of 64, and in comparison, Gaussian data augmentation and adversarial training achieve\nmCE of 98 and 108, respectively. In addition, as we can see, AutoAugment improves robustness on\nall but one of the corruptions, compared to the naturally trained model.\n\n4Our experiment is based on the open source implementation of AutoAugment at\n\nhttps://github.com/tensorflow/models/tree/master/research/autoaugment.\n\n7\n\n\u22120.10.00.10.20.30.40.50.60.70.8fraction of high frequency energy\u22120.2\u22120.10.00.10.20.30.4test acc - test acc(natural)Gauss, ave acc = 0.83adversarial, ave acc = 0.81low pass, ave acc = 0.83\u22120.10.00.10.20.30.40.50.60.70.8fraction of high frequency energy\u22120.20\u22120.15\u22120.10\u22120.050.00test acc - test acc(natural)high pass, ave acc = 0.68\u22120.10.00.10.20.30.40.50.60.70.8fraction of high frequency energy\u22120.2\u22120.10.00.10.20.30.4test acc - test acc(natural)AutoAug, ave acc = 0.86\fmodel\nnatural\nGauss\n\nadversarial\n\nAuto\n\nacc mCE speckle shot impulse defocus Gauss glass motion zoom snow fog bright contrast elastic pixel jpeg\n80\n77\n91\n83\n85\n81\n86\n81\n\n100\n98\n108\n64\n\n73\n90\n85\n71\n\n68\n92\n83\n78\n\n70\n92\n82\n81\n\n54\n83\n69\n86\n\n81\n77\n80\n85\n\n80\n82\n83\n90\n\n85\n88\n83\n89\n\n90\n72\n73\n95\n\n95\n92\n87\n96\n\n85\n84\n84\n92\n\n82\n57\n77\n95\n\n86\n84\n82\n87\n\nnoise\n\nweather\n\ndigital\n\nblur\n\n57\n80\n80\n76\n\n73\n79\n82\n88\n\nTable 2: Comprison between naturally trained model (natural), Gaussian data augmentation (Gauss), adversar-\nially trained model (adversarial), and AutoAugment (Auto) on CIFAR-10-C. We remove all corruptions that\nappear in this benchmark from the AutoAugment policy. All numbers are in percentage. The \ufb01rst column shows\nthe average top1 test accuracy on all the corruptions; the second column shows the mCE; the rest of the columns\nshow the average test accuracy over the 5 severities for each corruption. We observe that AutoAugment achieves\nthe best average test accuracy and the best mCE. In most of the blurring and all of the weather corruptions,\nAutoAugment achieves the best performance among the four models.\n\nAs for the ImageNet-C benchmark, instead of using the compressed ImageNet-C images provided\nin [17], we evaluate the models on corruptions applied in memory, 5 and observe that AutoAugment\nalso achieves the highest average corruption test accuracy. The full results can be found in the\nappendix. As for the compressed ImageNet-C images, we note that a follow-up work has utilized\nAutoAugment in a way to achieve state-of-the-art results [1].\n\n4.4 Adversarial examples are not strictly a high frequency phenomenon\nAdversarial perturbations remain a popular topic of study in the machine learning community. A\ncommon hypothesis is that adversarial perturbations lie primarily in the high frequency domain. In\nfact, several (unsuccessful) defenses have been proposed motivated speci\ufb01cally by this hypothesis.\nUnder the assumption that compression removes high frequency information, JPEG compression has\nbeen proposed several times [21, 2, 7] as a method for improving robustness to small perturbations.\nStudying the statistics of adversarially generated perturbations is not a well de\ufb01ned problem because\nthese statistics will ultimately depend on how the adversary constructs the perturbation. This dif\ufb01culty\nhas led to many false claims of methods for detecting adversarial perturbations [5]. Thus the analysis\npresented here is to better understand common hypothesis about adversarial perturbations, rather than\nactually detect all possible perturbations.\nFor several models we use PGD to construct adversarial perturbations for every image in the test set.\nWe then analyze the delta between the clean and perturbed images and project these deltas into the\nFourier domain. By aggregating across the successful attack images, we obtain an understanding\nof the frequency properties of the constructed adversarial perturbations. The results are shown in\nFigure 7.\nFor the naturally trained model, the measured adversarial perturbations do indeed show higher\nconcentrations in the high frequency domain (relative to the statistics of natural images). However,\nfor the adversarially trained model this is no longer the case. The deltas for the adversarially trained\nmodel resemble that of natural data. Our analysis provides some additional understanding on a\nnumber of observations in prior works on adversarial examples. First, while adversarial perturbations\nfor the naturally trained model do indeed show higher concentrations in the high frequency domain,\nthis does not mean that removing high frequency information from the input results in a robust model.\nIndeed as shown in Figure 3, the naturally trained model is not worst-case or even average-case robust\non any frequency (except perhaps the extreme low frequencies). Thus, we should expect that if we\nadversarially searched for errors in the low frequency domain, we will \ufb01nd them easily. This explains\nwhy JPEG compression, or any other method based on speci\ufb01cally removing high frequency content,\nshould not be expected to be robust to worst-case perturbations.\nSecond, the fact that adversarial training biases these perturbations towards the lower frequencies\nsuggests an intriguing connection between adversarial training and the DeepViz [23] method for\nfeature visualization. In particular, optimizing the input in the low frequency domain is one of the\nstrategies utilized by DeepViz to bias the optimization in the image space towards semantically\nmeaningful directions. Perhaps the reason adversarially trained models have semantically meaningful\ngradients [25] is because gradients are biased towards low frequencies in a similar manner as utilized\nin DeepViz.\n\n5The dataset of images with corruptions in memory can be found at https://github.com/tensorflow/\n\ndatasets/blob/master/tensorflow_datasets/image/imagenet2012_corrupted.py.\n\n8\n\n\fFigure 7: (a) and (b): Fourier spectrum of adversarial perturbations. For any image X, we run the PGD\nattack [22] to generate an adversarial example C(X). We estimate the Fourier spectrum of the adversarial\nperturbation, i.e., E[|F(C(X) \u2212 X)[i, j]|], where the expectation is taken over the perturbed images which\nare incorrectly classi\ufb01ed. (a) naturally trained; (b) adversarially trained. The adversarial perturbations for the\nnaturally trained model are uniformly distributed across frequency components. In comparison, adversarial\ntraining biases these perturbations towards the lower frequencies. (c) and (d): Adding Fourier basis vectors with\nlarge norm to images is a simple method for generating content-preserving black box adversarial examples.\n\nAs a \ufb01nal note, we observe that adding certain Fourier basis vectors with large norm (24 for ImageNet)\ndegrades test accuracy to less than 10% while preserving the semantics of the image. Two examples\nof the perturbed images are shown in Figure 7. If additional model queries are allowed, subtler\nperturbations will suf\ufb01ce \u2014 the perturbations used in Figure 4 can drop accuracies to less than 30%.\nThus, these Fourier basis corruptions can be considered as content-preserving black box attacks, and\ncould be of interest to research on security. Fourier heat maps with larger perturbations are included\nin the appendix.\n5 Conclusions and future work\nWe obtained a better understanding of trade-offs observed in recent robustness work in the image\ndomain. By investigating common corruptions and model performance in the frequency domain\nwe establish connections between frequency of a corruption and model performance under data\naugmentation. This connection is strongest for high frequency corruptions, where Gaussian data\naugmentation and adversarial training bias the model towards low frequency information in the input.\nThis results in improved robustness to corruptions with higher concentrations in the high frequency\ndomain at the cost of reduced robustness to low frequency corruptions and clean test error.\nSolving the robustness problem via data augmentation alone feels quite challenging given the trade-\noffs we commonly observe. Naively augmenting on different corruptions often will not transfer well\nto held out corruptions [12]. However, the impressive robustness of AutoAugment gives us hope that\ndata augmentation done properly can play a crucial role in mitigating the robustness problem.\nCare must be taken though when utilizing data augmentation for robustness to not over\ufb01t to the\nvalidation set of held out corruptions. The goal is to learn domain invariant features rather than simply\nbecome robust to a speci\ufb01c set of corruptions. The fact that AutoAugment was tuned speci\ufb01cally\nfor clean test error, and transfers well even after removing the contrast and brightness parts of the\npolicy (as these corruptions appear in the benchmark) gives us hope that this is a step towards more\nuseful domain invariant features. The robustness problem is certainly far from solved, and our Fourier\nanalysis shows that the AutoAugment model is not strictly more robust than the baseline \u2014 there are\nfrequencies for which robustness is degraded rather than improved. Because of this, we anticipate\nthat robustness benchmarks will need to evolve over time as progress is made. These trade-offs are to\nbe expected and researchers should actively search for new blindspots induced by the methods they\nintroduce. As we grow in our understanding of these trade-offs we can design better benchmarks to\nobtain a more comprehensive perspective on model robustness.\nWhile data augmentation is perhaps the most effective method we currently have for the robustness\nproblem, it seems unlikely that data augmentation alone will provide a complete solution. Towards\nthat end it will be important to develop orthogonal methods \u2014 e.g. architectures with better inductive\nbiases or loss functions which when combined with data augmentation encourage extrapolation rather\nthan interpolation.\nAcknowledgments\nWe would like to thank Nicolas Ford and Norman Mu for helpful discussions.\n\n9\n\n\fReferences\n[1] Anonymous. AugMix: A simple method to improve robustness and uncertainty under data shift.\n\nIn Submitted to International Conference on Learning Representations, 2020. under review.\n\n[2] A. E. Aydemir, A. Temizel, and T. T. Temizel. The effects of JPEG and JPEG2000 compression\n\non attacks using adversarial examples. 2018.\n\n[3] A. Azulay and Y. Weiss. Why do deep convolutional networks generalize so poorly to small\n\nimage transformations? arXiv preprint arXiv:1805.12177, 2018.\n\n[4] R. N. Bracewell and R. N. Bracewell. The Fourier transform and its applications, volume\n\n31999. McGraw-Hill New York, 1986.\n\n[5] N. Carlini and D. Wagner. Adversarial examples are not easily detected: Bypassing ten detection\nmethods. In Proceedings of the 10th ACM Workshop on Arti\ufb01cial Intelligence and Security,\npages 3\u201314. ACM, 2017.\n\n[6] E. D. Cubuk, B. Zoph, D. Mane, V. Vasudevan, and Q. V. Le. Autoaugment: Learning\naugmentation policies from data. In Proceedings of the IEEE Conference on Computer Vision\nand Pattern Recognition (CVPR), pages 113\u2013123, 2019.\n\n[7] N. Das, M. Shanbhogue, S.-T. Chen, F. Hohman, S. Li, L. Chen, M. E. Kounavis, and D. H.\nChau. Shield: Fast, practical defense and vaccination for deep learning using jpeg compression.\nIn Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery\nand Data Mining (KDD), pages 196\u2013204, 2018.\n\n[8] S. Dodge and L. Karam. A study and comparison of human and deep learning recognition\nperformance under visual distortions. In 2017 26th International Conference on Computer\nCommunication and Networks (ICCCN), pages 1\u20137. IEEE, 2017.\n\n[9] A. Dziedzic, J. Paparrizos, S. Krishnan, A. Elmore, and M. Franklin. Band-limited training\nand inference for convolutional neural networks. In Proceedings of the 36th International\nConference on Machine Learning (ICML), pages 1745\u20131754, 2019.\n\n[10] N. Ford, J. Gilmer, N. Carlini, and E. D. Cubuk. Adversarial examples are a natural consequence\nof test error in noise. In Proceedings of the 36th International Conference on Machine Learning\n(ICML), pages 2280\u20132289, 2019.\n\n[11] R. Geirhos, P. Rubisch, C. Michaelis, M. Bethge, F. A. Wichmann, and W. Brendel. ImageNet-\ntrained CNNs are biased towards texture; increasing shape bias improves accuracy and robust-\nness. In Proceedings of the International Conference on Learning Representations (ICLR),\n2019.\n\n[12] R. Geirhos, C. R. Temme, J. Rauber, H. H. Sch\u00fctt, M. Bethge, and F. A. Wichmann. General-\nisation in humans and deep neural networks. In Advances in Neural Information Processing\nSystems (NeurIPS), pages 7538\u20137550, 2018.\n\n[13] J. Gilmer, R. P. Adams, I. Goodfellow, D. Andersen, and G. E. Dahl. Motivating the rules of the\n\ngame for adversarial example research. arXiv preprint arXiv:1807.06732, 2018.\n\n[14] I. J. Goodfellow, J. Shlens, and C. Szegedy. Explaining and harnessing adversarial examples. In\n\nProceedings of the International Conference on Learning Representations (ICLR), 2015.\n\n[15] C. Guo, J. R. Gardner, Y. You, A. G. Wilson, and K. Q. Weinberger. Simple black-box\n\nadversarial attacks. arXiv preprint arXiv:1905.07121, 2019.\n\n[16] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition.\n\nIn\nProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR),\npages 770\u2013778, 2016.\n\n[17] D. Hendrycks and T. Dietterich. Benchmarking neural network robustness to common cor-\nIn Proceedings of the International Conference on Learning\n\nruptions and perturbations.\nRepresentations (ICLR), 2019.\n\n[18] A. Ilyas, S. Santurkar, D. Tsipras, L. Engstrom, B. Tran, and A. Madry. Adversarial examples\n\nare not bugs, they are features. arXiv preprint arXiv:1905.02175, 2019.\n\n[19] J.-H. Jacobsen, J. Behrmann, R. Zemel, and M. Bethge. Excessive invariance causes adversarial\nvulnerability. In Proceedings of the International Conference on Learning Representations\n(ICLR), 2018.\n\n10\n\n\f[20] J. Jo and Y. Bengio. Measuring the tendency of CNNs to learn surface statistical regularities.\n\narXiv preprint arXiv:1711.11561, 2017.\n\n[21] Z. Liu, Q. Liu, T. Liu, Y. Wang, and W. Wen. Feature distillation: DNN-oriented JPEG\ncompression against adversarial examples. In Proceedings of the IEEE Conference on Computer\nVision and Pattern Recognition (CVPR), pages 860\u2013868, 2019.\n\n[22] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu. Towards deep learning models\nresistant to adversarial attacks. In Proceedings of the International Conference on Learning\nRepresentations (ICLR), 2018.\n\n[23] C. Olah, A. Mordvintsev, and L. Schubert.\n\nhttps://distill.pub/2017/feature-visualization.\n\nFeature visualization.\n\nDistill, 2017.\n\n[24] B. Recht, R. Roelofs, L. Schmidt, and V. Shankar. Do ImageNet classi\ufb01ers generalize to\nImageNet? In Proceedings of the 36th International Conference on Machine Learning (ICML),\npages 5389\u20135400, 2019.\n\n[25] D. Tsipras, S. Santurkar, L. Engstrom, A. Turner, and A. Madry. Robustness may be at odds\nwith accuracy. In Proceedings of the International Conference on Learning Representations\n(ICLR), 2019.\n\n[26] Y. Tsuzuku and I. Sato. On the structural sensitivity of deep convolutional networks to the\ndirections of fourier basis functions. In Proceedings of the IEEE Conference on Computer\nVision and Pattern Recognition (CVPR), pages 51\u201360, 2019.\n\n[27] S. Zagoruyko and N. Komodakis. Wide residual networks. In Proceedings of the British\n\nMachine Vision Conference (BMVC), pages 87.1\u201387.12, 2016.\n\n11\n\n\f", "award": [], "sourceid": 7280, "authors": [{"given_name": "Dong", "family_name": "Yin", "institution": "UC Berkeley"}, {"given_name": "Raphael", "family_name": "Gontijo Lopes", "institution": "Google Brain"}, {"given_name": "Jon", "family_name": "Shlens", "institution": "Google Research"}, {"given_name": "Ekin Dogus", "family_name": "Cubuk", "institution": "Google Brain"}, {"given_name": "Justin", "family_name": "Gilmer", "institution": "Google Brain"}]}