{"title": "e-SNLI: Natural Language Inference with Natural Language Explanations", "book": "Advances in Neural Information Processing Systems", "page_first": 9539, "page_last": 9549, "abstract": "In order for machine learning to garner widespread public adoption, models must be able to provide interpretable and robust explanations for their decisions, as well as learn from human-provided explanations at train time. In this work, we extend the Stanford Natural Language Inference dataset with an additional layer of human-annotated natural language explanations of the entailment relations. We further implement models that incorporate these explanations into their training process and output them at test time. We show how our corpus of explanations, which we call e-SNLI, can be used for various goals, such as obtaining full sentence justifications of a model\u2019s decisions, improving universal sentence representations and transferring to out-of-domain NLI datasets. Our dataset thus opens up a range of research directions for using natural language explanations, both for improving models and for asserting their trust", "full_text": "e-SNLI: Natural Language Inference with\n\nNatural Language Explanations\n\nOana-Maria Camburu1 Tim Rockt\u00e4schel2 Thomas Lukasiewicz1,3 Phil Blunsom1,4\n\n{oana-maria.camburu, thomas.lukasiewicz, phil.blunsom}@cs.ox.ac.uk\n\nt.rocktaschel@ucl.ac.uk\n\n1Department of Computer Science, University of Oxford\n\n2Department of Computer Science, University College London\n\n3Alan Turing Institute, London, UK\n\n4DeepMind, London, UK\n\nAbstract\n\nIn order for machine learning to garner widespread public adoption, models must\nbe able to provide interpretable and robust explanations for their decisions, as\nwell as learn from human-provided explanations at train time. In this work, we\nextend the Stanford Natural Language Inference dataset with an additional layer of\nhuman-annotated natural language explanations of the entailment relations. We\nfurther implement models that incorporate these explanations into their training\nprocess and output them at test time. We show how our corpus of explanations,\nwhich we call e-SNLI, can be used for various goals, such as obtaining full sentence\njusti\ufb01cations of a model\u2019s decisions, improving universal sentence representations\nand transferring to out-of-domain NLI datasets. Our dataset1 thus opens up a range\nof research directions for using natural language explanations, both for improving\nmodels and for asserting their trust.\n\n1\n\nIntroduction\n\nHumans do not learn solely from labeled examples supplied by a teacher. Instead, they seek a\nconceptual understanding of a task through both demonstrations and explanations. Machine learning\nmodels trained simply to obtain high accuracy on held-out sets often learn to rely heavily on shallow\ninput statistics, resulting in brittle models susceptible to adversarial attacks. For example, Ribeiro\net al. [24] present a document classi\ufb01er that distinguishes between Christianity and Atheism with an\naccuracy of 94%. However, on close inspection, the model spuriously separates classes based on\nwords contained in the headers, such as Posting, Host, and Re.\n\nIn this work, we introduce a new dataset and models for exploiting and generating explanations for\nthe task of recognizing textual entailment. We argue for free-form natural language explanations, as\nopposed to formal language, for a series of reasons. First, natural language is readily comprehensible\nto an end-user who needs to assert a model\u2019s reliability. Secondly, it is also easiest for humans to\nprovide free-form language, eliminating the additional effort of learning to produce formal language,\nthus making it simpler to collect such datasets. Lastly, natural language justi\ufb01cations might eventually\nbe mined from existing large-scale free-form text.\n\nDespite the potential for free-form justi\ufb01cations to improve both learning and transparency, there is\ncurrently a lack of such datasets in the machine learning community. To address this de\ufb01ciency, we\nhave collected a large corpus of human-annotated explanations for the Stanford Natural Language\n\n1https://github.com/OanaMariaCamburu/e-SNLI\n\n32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Montr\u00e9al, Canada.\n\n\fPremise: An adult dressed in black holds a stick.\nHypothesis: An adult is walking away, empty-handed.\nLabel: contradiction\nExplanation: Holds a stick implies using hands so it is not empty-handed.\n\nPremise: A child in a yellow plastic safety swing is laughing as a dark-haired woman\nin pink and coral pants stands behind her.\nHypothesis: A young mother is playing with her daughter in a swing.\nLabel: neutral\nExplanation: Child does not imply daughter and woman does not imply mother.\n\nPremise: A man in an orange vest leans over a pickup truck.\nHypothesis: A man is touching a truck.\nLabel: entailment\nExplanation: Man leans over a pickup truck implies that he is touching it.\n\nFigure 1: Examples from e-SNLI. Annotators were given the premise, hypothesis, and label. They\nhighlighted the words that they considered essential for the label and provided the explanations.\n\nInference (SNLI) dataset [3]. We chose SNLI because it constitutes an in\ufb02uential corpus for natural\nlanguage understanding that requires deep assimilation of \ufb01ne-grained nuances of common-sense\nknowledge. We call our explanation-augmented dataset e-SNLI, which we collected to enable\nresearch in the direction of training with and generation of free-form textual justi\ufb01cations.\n\nIn order to demonstrate the ef\ufb01cacy of the e-SNLI dataset, we \ufb01rst show that it is much more dif\ufb01cult\nto produce correct explanations based on spurious correlations than to produce correct labels. We\nthen implement models that, given a premise and a hypothesis, predict a label and an explanation. We\nalso investigate how the additional signal from explanations received at train time can guide models\ninto learning better sentence representations. Finally, we look into the transfer capabilities of our\nmodel to out-of-domain NLI datasets.\n\n2 Background\n\nThe task of recognizing textual entailment is a critical natural language understanding task. Given a\npair of sentences, called the premise and hypothesis, the task consists of classifying their relation\nas either (a) entailment, if the premise entails the hypothesis, (b) contradiction, if the hypothesis\ncontradicts the premise, or (c) neutral, if neither entailment nor contradiction hold. The SNLI dataset\n[3], containing 570K data points of human-generated triples (premise, hypothesis, label), has driven\nthe development of a large number of neural network models [25, 21, 22, 6, 19, 5, 7].\n\nConneau et al. [7] showed that training universal sentence representations on SNLI is both more\nef\ufb01cient and more accurate than the traditional training approaches on orders of magnitude larger, but\nunsupervised, datasets [17, 14]. We take this approach one step further and show that an additional\nlayer of explanations on top of the label supervision brings further improvement.\n\nRecently, Gururangan et al. [13] cast doubt on whether models trained on SNLI are learning to\nunderstand language, or are largely \ufb01xating on spurious correlations, also called artifacts. For\nexample, speci\ufb01c words in the hypothesis tend to be strong indicators of the label, e.g., friends, old\nappear very often in neutral hypotheses, animal, outdoors appear most of the time in entailment\nhypotheses, while nobody, sleeping appear mostly in contradiction hypothesis. They show that a\npremise-agnostic model, i.e., a model that only takes as input the hypothesis and outputs the label,\nobtains 67% test accuracy. In section 4.1 we show that it is much more dif\ufb01cult to rely on artifacts to\ngenerate explanations than to generate labels.\n\n3 Collecting explanations\n\nWe present our collection methodology for e-SNLI, for which we used Amazon Mechanical Turk.\nThe main question that we want our dataset to answer is: Why is a pair of sentences in a relation of\nentailment, neutrality, or contradiction? We encouraged the annotators to focus on the non-obvious\n\n2\n\n\felements that induce the given relation, and not on the parts of the premise that are repeated identically\nin the hypothesis. For entailment, we required justi\ufb01cations of all the parts of the hypothesis that do\nnot appear in the premise. For neutral and contradictory pairs, while we encouraged stating all the\nelements that contribute to the relation, we consider an explanation correct, if at least one element\nis stated. Finally, we asked the annotators to provide self-contained explanations, as opposed to\nsentences that would make sense only after reading the premise and hypothesis. For example, we\nprefer an explanation of the form \u201cAnyone can knit, not just women.\u201d, rather than \u201cIt cannot be\ninferred they are women.\u201d\n\nIn crowd-sourcing, it is dif\ufb01cult to control the quality of free-form annotations. Thus, we aimed\nto preemptively block the submission of obviously incorrect answers. We did in-browser checks\nto ensure that each explanation contained at least three tokens and that it was not a copy of the\npremise or hypothesis. We further guided the annotators to provide adequate answers by asking\nthem to proceed in two steps. First, we require them to highlight words from the premise and/or\nhypothesis that they consider essential for the given relation. Secondly, annotators had to formulate\nthe explanation using the words that they highlighted. However, using exact spelling might push\nannotators to formulate grammatically incorrect sentences, therefore we only required half of the\nhighlighted words to be used with the same spelling. For entailment pairs, we required at least one\nword in the premise to be highlighted. For contradiction pairs, we required highlighting at least one\nword in both the premise and the hypothesis. For neutral pairs, we only allowed highlighting words in\nthe hypothesis, in order to strongly emphasize the asymmetry in this relation and to prevent workers\nfrom confusing the premise with the hypothesis. We believe these label-speci\ufb01c constraints helped in\nputting the annotator into the correct mindset, and additionally gave us a means to \ufb01lter incorrect\nexplanations. Finally, we also checked that the annotators used other words that were not highlighted,\nas we believe a correct explanation would need to articulate a link between the keywords.\n\nWe collected one explanation for each pair in the training set and three explanations for each pair in\nthe validation and test sets. Figure 1 shows examples of collected explanations. There were 6325\nworkers with an average of 86 explanations per worker and a standard deviation of 403.\n\nAnalysis and re\ufb01nement of the collected dataset In order to measure the quality of our collected\nexplanations, we selected a random sample of 1000 examples and manually graded their correctness\nbetween 0 (incorrect) and 1 (correct), giving partial scores of k/n if only k out of n required\narguments were mentioned. We also considered an explanation as incorrect if it was uninformative,\nthat is, if the explanation was template-like, extensively repeating details from the premise/hypothesis\nthat are not directly useful for justifying the relation between the two sentences. We observed a few\nre-occurring templates such as: \u201cJust because [entire premise] doesn\u2019t mean [entire hypothesis]\u201d for\nneutral pairs, \u201c[entire premise] implies [entire hypothesis]\u201d for entailment pairs, and \u201cIt can either be\n[entire premise] or [entire hypothesis]\u201d for contradiction pairs. We assembled a list of templates,\nwhich can be found in Appendix A, that we used for \ufb01ltering the dataset of such uninformative\nexplanations. Speci\ufb01cally, we \ufb01ltered an explanation if its edit distance to one of the templates\nwas less than 10. We ran this template detection on the entire dataset and reannotated the detected\nexplanations (11% in total).\n\nOur \ufb01nal counts show a total error rate of 9.62%, with 19.55% on entailment, 7.26% on neutral,\nand 9.38% on contradiction. We notice that entailment pairs were by far the most dif\ufb01cult to obtain\nproper explanations for. This is \ufb01rstly due to partial explanations, as annotators had an incentive to\nprovide shorter inputs, so they often only mentioned one argument. A second reason is that many of\nthe entailment pairs have the hypothesis as almost a subset of the premise, prompting the annotators\nto just repeat that as a statement.\n\n4 Experiments\n\nWe \ufb01rst present an experiment which demonstrates that a model which can easily rely on artifacts in\nSNLI to provide correct labels would not be able to provide correct explanations as easily. We refer\nto it as PREMISEAGNOSTIC.\n\nWe then present a series of experiments to elucidate whether models trained on e-SNLI are able to:\n(i) predict a label and generate an explanation for the predicted label (referred to as PREDICTAND-\nEXPLAIN), (ii) generate an explanation then predict the label given only the generated explanation\n\n3\n\n\f(EXPLAINTHENPREDICT), (iii) learn better universal sentence representations (REPRESENT), and\n(iv) transfer to out-of-domain NLI datasets (TRANSFER).\n\nThroughout our experiments, our models follow the architecture presented in Conneau et al. [7], as\nwe build directly on top of their code2. Therefore, our encoders are 2048-bidirectional-LSTMs [15]\nwith max-pooling, resulting in a sentence representation dimension of 4096. Our label classi\ufb01ers\nare 3-layers MLPs with 512 internal size and without non-linearities. For our explanation decoders,\nwe used a simple one-layer LSTM, for which we tried internal sizes of 512, 1024, 2048, and 4096.\nIn order to reduce the vocabulary size for explanation generation, we replaced words that appeared\nless than 15 times3 with <UNK>. We obtain an output vocabulary of approximately 12K words. The\npreprocessing and optimization were kept the same as in [7].\n\nWhenever appropriate, we run our models with \ufb01ve seeds and provide the average performance with\nthe standard deviation in parenthesis. If no standard deviation is reported, the results are from one\nexperiment with seed 1234.\n\n4.1 PREMISEAGNOSTIC: Generate an explanation given only the hypothesis\n\nGururangan et al. [13] show that a neural network that only has access to the hypothesis can predict\nthe correct label 67% of the times. We are therefore interested in evaluating how well our explanations\ncan be predicted from hypotheses alone.\n\nModel We train a 2048-bidirectional-LSTM with max-pooling for encoding the hypothesis, followed\nby a one-layer LSTM for decoding the explanation. The initial state of the decoder is the vector\nembedding of the hypothesis, which is also concatenated at every timestep of the decoder, to avoid\nforgetting.\n\nSelection We consider internal sizes of the decoder of 512, 1024, 2048 and 4096. We pick the model\nthat gives the best perplexity on the validation set. We notice that the perplexity strictly decreases\nwhen we increase the decoder size. However, for practical reasons, we do not increase the decoder\nsize beyond 4096.\n\nResults We then manually look at the \ufb01rst 100 test examples and obtain that only 6.834 were correct.\nWe also separately train the same hypothesis-only encoder for label prediction alone and obtain\n66 correct labels in the same \ufb01rst 100 test examples. This validates our intuition that it is much\nmore dif\ufb01cult (approx. 10x for this architecture) to rely on spurious correlations to predict correct\nexplanations than to predict correct labels.\n\n4.2 PREDICTANDEXPLAIN: Jointly predict a label and generate an explanation for the\n\npredicted label\n\nIn this experiment, we investigate how the typical architecture employed on SNLI can be enhanced\nwith a module that aims to justify the decisions of the entire network.\n\nModel We employ the InferSent [7] architecture, where a bidirectional-LSTM with max-pooling\nseparately encodes the premise, u, and hypothesis, v. The vector of features f = [u, v, |u\u2212v|, u\u2299v]\nis then passed to the MLP classi\ufb01er that outputs a distribution over the 3 labels. We add a one-layer\nLSTM decoder for explanations, which takes the feature vector f both as an initial state and\nconcatenated to the word embedding at each time step.\n\nIn order to condition the explanation also on the label, we prepend the label as a word (entailment,\ncontradiction, neutral) at the beginning of the explanation. At training time, the gold label is provided,\nwhile at test time, we use the label predicted by the classi\ufb01er. This architecture is depicted in Figure 2.\n\nLoss We use negative log-likelihood for both classi\ufb01cation and explanation losses. The explanation\nloss is much larger in magnitude than the classi\ufb01cation loss, due to the summation of negative\nlog-likelihoods over the words in the explanations. To account for this difference during training, we\n\n2https://github.com/facebookresearch/InferSent. We \ufb01xed the issue raised in https://github.com/ facebookre-\n\nsearch/InferSent/issues/51 that the max-pooling was taken over paddings.\n\n3Counted among premises, hypothesis, and explanations.\n4Partial scoring as explained in Section 3.\n\n4\n\n\fuse a weighting coef\ufb01cient \u03b1 \u2208 [0, 1]. Hence, our overall loss is:\n\nLtotal = \u03b1Llabel + (1 \u2212 \u03b1)Lexplanation\n\n(1)\n\nSelection We consider \u03b1 values from 0.1 to 0.9\nwith a step of 0.1 and decoder internal sizes of\n512, 1024, 2048, and 4096. For this experiment,\nwe choose as model selection criterion the ac-\ncuracy on the SNLI validation set, because we\nwant to investigate how well a model can gen-\nerate justi\ufb01cations without sacri\ufb01cing accuracy.\nAs future work, one can inspect different trade-\noffs between accuracy and explanation genera-\ntion. We found \u03b1 = 0.6 and the decoder size of\n512 to produce the best validation accuracy, of\n84.37%, while InferSent with no explanations\nproduced 84.30% validation accuracy. We call\nour model e-INFERSENT, since it freezes the In-\nferSent architecture and training procedure, and\nonly adds the explanations decoder.\n\nLabel\n\nf\n\nExplanation\n\nPremise\n\nHypothesis\n\nFigure 2: Overview of the e-INFERSENT architec-\nture.\n\nResults The average test accuracy that we obtain when training InferSent[7] on SNLI with \ufb01ve seeds\nis 84.01% (0.25). Our e-INFERSENT model obtains essentially the same test accuracy, of 83.96%\n(0.26), which shows that one can get additional justi\ufb01cations without sacri\ufb01cing label accuracy. For\nthe generated explanations, we obtain a perplexity of 10.58(0.4) and a BLEU-score of 22.40(0.7).\nSince we collected 3 explanations for each example in the validation and test sets, we compute\nthe inter-annotator BLEU-score of the third explanation with respect to the \ufb01rst two, and obtain\n22.51. For consistency, we used the same two explanations as the only references when computing\nthe BLEU-score for the predicted explanations. Given the low inter-annotator score and the fact\nthat generated explanations almost match the inter-annotator BLEU-score, we conclude that this\nmeasure is not reliable for our task, and we further rely on human evaluation. Therefore, we manually\nannotated the \ufb01rst 100 datapoints in the test set (we used the same partial scoring as in Section 3).\nSince the explanation is conditioned on the predicted label, for incorrect labels, the model would not\nproduce a correct explanation. Therefore, we provide as correctness score the percentage of correct\nexplanations in the subset of the \ufb01rst 100 examples where the predicted label was correct (80 in this\nexperiment). We obtain a percentage of 34.68% correct explanations. While this percentage is low,\nwe keep in mind that the selection criteria was only the accuracy of the label classi\ufb01er and not the\nperplexity of the explanation. In the next experiment, we show how training (and selecting) only for\ngenerating explanations results in higher quality explanations.\n\n4.3 EXPLAINTHENPREDICT: Generate an explanation then predict a label\n\nIn PREDICTANDEXPLAIN, we conditioned the explanation on the label predicted by the MLP,\nbecause we wanted to see how the typical architecture used on SNLI can be adapted to justify its\ndecisions in natural language. However, a more natural approach for solving inference is to think of\nthe explanation \ufb01rst and based on the explanation to decide a label. Therefore, in this experiment, we\n\ufb01rst train a network to generate an explanation given a pair of (premise, hypothesis), and, separately,\nwe train a network to provide a label given an explanation. This is a sensible decomposition for our\ndataset, due to the following key observation: In our dataset, in the large majority of the cases, one\ncan easily detect for which label an explanation has been provided. We highlight that this is not the\ncase in general, as the same explanation can be correctly arguing for different labels, depending on\nthe premise and hypothesis. For example, the explanation \"A woman is a person\" would be a correct\nexplanation for the entailment pair (\"A woman is in the park\", \"A person is in the park\") as well\nfor the contradiction pair (\"A woman is in the park\", \"There is no person in the park\"). However,\nthere are multiple ways of formulating an explanation. In our example, for the contradiction pair,\none could also explain that \"There cannot be no person in the park if a woman is in the park\", which\nread alone would allow one to infer that the pair was a contradiction. To support our observation, we\ntrain a neural network that given only an explanation predicts a label. We use the same bidirectional\nencoder and MLP-classi\ufb01er as above. We obtain an accuracy of 96.83% on the test set of SNLI.\n\n5\n\n\fTable 1: Summary of the performance of the models PREDICTANDEXPLAIN, EXPLAINTHENPRE-\nDICT. The averages are over \ufb01ve seeds and the standard deviations are in parenthesis. The baseline\nInferSent-SNLI-ours is the InferSent model [7] that we re-run with 5 seeds. ExplCorrect@100 is the\nscore of correctness for the generated explanations, that we manually annotated for the \ufb01rst 100 data\npoints in the SNLI test set for one seed.\n\nModel\n\nLabel Accuracy\n\nPerplexity\n\nBLEU\n\nExplCorrect@100\n\nInferSent-SNLI-ours\ne-INFERSENT\nEXPLAINTHENPREDICTSEQ2SEQ\nEXPLAINTHENPREDICTATTENTION\n\n84.01 (0.25)\n83.96 (0.26)\n81.59 (0.45)\n81.71 (0.36)\n\n-\n10.58 (0.4)\n8.95 (0.03)\n6.1 (0)\n\n-\n22.40 (0.7)\n24.14 (0.58)\n27.58 (0.47)\n\n-\n34.68\n49.8\n64.27\n\nModels For predicting an explanation given a pair of (premise, hypothesis), we \ufb01rst train a simple\nseq2seq model that we call EXPLAINTHENPREDICTSEQ2SEQ. Essentially, we keep the architecture\nin e-INFERSENT, where we eliminate the classi\ufb01er by setting \u03b1 = 0, and we decode the expla-\nnation without prepending the label. Secondly, we train an attention model, which we refer to as\nEXPLAINTHENPREDICTATTENTION. Attention mechanisms in neural networks brought consistent\nimprovements over the non-attention counter-parts in various areas, such as computer vision [27],\nspeech [4], or natural language processing [12, 2]. We use the same encoder and decoder as in\nEXPLAINTHENPREDICTSEQ2SEQ, and we add two identical but separate attention modules, over\nthe tokens in the premise and hypothesis. For details of the attention modules, see Appendix B.\n\nSelection Our only hyper-parameter is internal sizes for the decoder of 512, 1024, 2048, and 4096.\nOur model selection criterion is the perplexity on the validation set of SNLI. We obtain the best\ncon\ufb01guration for both EXPLAINTHENPREDICTSEQ2SEQ and EXPLAINTHENPREDICTATTENTION\nto have an internal size of 1024.\n\nResults With the described setup, the SNLI test accuracy drops from 83.96% (0.26) in PRE-\nDICTANDEXPLAIN to 81.59% (0.45) in EXPLAINTHENPREDICTSEQ2SEQ and 81.71%(0.36) in\nEXPLAINTHENPREDICTATTENTION. However, when we again manually annotate the \ufb01rst 100\ngenerated explanations in the test set, we obtain signi\ufb01cantly higher percentages of correct explana-\ntions: 49.8% for EXPLAINTHENPREDICTSEQ2SEQ and 64.27% for EXPLAINTHENPREDICTAT-\nTENTION. We note that the attention mechanism indeed signi\ufb01cantly increases the quality of the\nexplanations. The perplexity and BLEU-score are 8.95(0.03) and 24.14(0.58) for EXPLAINTHEN-\nPREDICTSEQ2SEQ, and 6.1(0) and 27.58(0.47) for EXPLAINTHENPREDICTATTENTION. Our\nexperiment shows that, while sacri\ufb01cing a bit of performance, we get a better trust that when EX-\nPLAINTHENPREDICT predicts a correct label, it does so for the right reasons. We summarize the\nresults in table 1.\n\nQualitative analysis of explanations In Table 2, we provide examples of generated explanations\nfrom the test set from (a) PREDICTANDEXPLAIN, (b) EXPLAINTHENPREDICTSEQ2SEQ, and (c)\nEXPLAINTHENPREDICTATTENTION. At the end of each explanation, we give in brackets the score\nthat we manually allocated as explained in section 3. We notice that the explanations are mainly on\ntopic for all the three models, with minor exceptions, such as the mention of \"camou\ufb02age\" in (1c).\nWe also notice that even when incorrect, they are sometimes frustratingly close to being correct, for\nexample, explanation (2b) is only one word (out of its 20 words) away from being correct. It is also\ninteresting to inspect the explanations provided when the predicted label is incorrect. For example,\nin (1a), we see that the network omitted the information of \"facing the camera\" in the premise and\ntherefore classi\ufb01ed the pair as neutral, which is backed up by an otherwise correct explanation in\nitself. We also see that model EXPLAINTHENPREDICTSEQ2SEQ correctly classi\ufb01es this pair as\nentailment, however, it only motivates 1 out the 3 reasons why it is so, and it also picks arguably the\neasiest reason. Interestingly, the attention model (1c) points to the correct evidence but argues that\n\"standing\" and \"facing a camera\" is not enough to conclude \"posing for a picture\".\n\n4.4 REPRESENT: Universal sentence representations\n\nObtaining universal sentence representations is the task of learning an encoder to provide semantically\nmeaningful \ufb01xed-length representations of phrases/sentences. These representations are further used\nas features in other downstream tasks, particularly for tasks where labeled training data are scarce. In\ncomputer vision, pretrained ImageNet-based encoders [10] provide standard image feature extractors\n\n6\n\n\fTable 2: Examples of predicted labels and generated explanations from (a) PREDICTANDEXPLAIN,\n(b) EXPLAINTHENPREDICTSEQ2SEQ, and (c) EXPLAINTHENPREDICTATTENTION. In brackets\nour scores of correctness.\n\n(1) Premise: 3 young man in hoods standing in the middle of a quiet street facing the camera.\n\nHypothesis: Three hood wearing people pose for a picture.\n\nGold label: entailment\n\n(a) Predicted label: neutral\nExplanation: Just because the men are in the\nmiddle of a street doesn\u2019t mean\nthey are posing for a picture. [0]\n\n(b) Predicted label: entailment\nExplanation: three young men\nare people. [0.33]\n\n(c) Predicted label: neutral\nExplanation: Just because three young man\nin camou\ufb02age standing in the middle of\na quiet street facing the camera\ndoes not mean they pose for a picture. [0]\n\n(2) Premise: Three \ufb01re\ufb01ghter come out of subway station.\n\nHypothesis: Three \ufb01re\ufb01ghters putting out a \ufb01re inside of a subway station.\n\nGold label: neutral\n\n(a) Predicted label: contradiction\nExplanation: The \ufb01re\ufb01ghters\ncan not be putting out a \ufb01re station and\nputting out a \ufb01re at the same time. [0]\n\n(b) Predicted label: neutral\nExplanation: The fact that three\n\ufb01remen are putting out of a subway\nstation doesn\u2019t imply that they\nare putting out a \ufb01re. [0]\n\n(c) Predicted label: neutral\nExplanation: The \ufb01re\ufb01ghters\nmay not be putting out a \ufb01re\ninside of the subway station. [1]\n\n(3) Premise: A blond-haired doctor and her African American assistant looking threw new medical manuals.\n\nHypothesis: A man is eating pb and j.\n\nGold label: contradiction\n\n(a) Predicted label: contradiction\nExplanation: A man is not a woman. [1]\n\n(b) Predicted label: contradiction\nExplanation: One can not be looking\nand eating simultaneously. [0]\n\n(c) Predicted label: contradiction\nExplanation: A person can not be looking\nat a medical and a book at the same time. [0]\n\nfor other downstream tasks. However, in natural language processing, there is still no consensus on\ngeneral-purpose sentence encoders. It remains an open question on which task and dataset should such\nan encoder be trained. Traditional approaches make use of very large unsupervised datasets [17, 14],\ntaking weeks to train. Conneau et al. [7] showed that training only on NLI is both more accurate\nand more time-ef\ufb01cient than training on orders of magnitude larger but unsupervised datasets. Their\nresults constitute a previous state-of-the-art for universal sentence representations and encourage the\nidea that supervision can be more bene\ufb01cial than larger but unsupervised datasets. We hypothesize\nthat an additional layer of supervision in the form of natural language explanations should further\nimprove learning of universal sentence representations.\n\nModel We use our e-INFERSENT model already trained in PREDICTANDEXPLAIN. While we\ncompare our model with InferSent that has not been trained on explanations, we want to ensure\nthat eventual improvements are not purely due to the addition of a language model in the decoder\nnetwork. We therefore introduce a second baseline, INFERSENTAUTOENC, where instead of decoding\nexplanations, we decode the premise and hypothesis separately from each sentence representation\nusing one shared decoder.\n\nEvaluation metrics Typically, sentence representations are evaluated by using them as \ufb01xed fea-\ntures on top of which shallow classi\ufb01ers are trained for a series of downstream tasks. Conneau\net al. [7] provide an excellent tool, called SentEval, for evaluating sentence representations on 10\ndiverse tasks: movie reviews (MR), product reviews (CR), subjectivity/objectivity (SUBJ), opinion\npolarity (MPQA), question-type (TREC), sentiment analysis (SST), semantic textual similarity\n(STS), paraphrase detection (MRPC), entailment (SICK-E), and semantic relatedness (SICK-R).\nWe refer to their work for a more detailed description of each of these tasks and of SentEval, which\nwe use for comparing the quality of the sentence embeddings obtained by additionally providing our\nexplanations on top of the label supervision.\n\nResults In Table 3, we report the average results and standard deviations of e-INFERSENT, our\nretrained InferSent model, and the additional INFERSENTAUTOENC baseline on the downstream\ntasks mentioned above. To test if the differences in performance of INFERSENTAUTOENC and\ne-INFERSENT relative to the InferSent baseline are signi\ufb01cant, we performed Welch\u2019s t-test.5 We\nmark with * the results that appeared signi\ufb01cant under the signi\ufb01cance level of 0.05.\n\n5Using the implementation in scipy.stats.ttest_ind with equal_var=False.\n\n7\n\n\fTable 3: Transfer results on downstream tasks. For MRPC we report accuracy/F1 score, for STS14\nwe report the Person/Spearman correlations, for SICK-R the Person correlation, and for all the rest\ntheir accuracies. Results are the average of 5 runs with different seeds. The standard deviations\nis shown in brackets, and the best result for every task is indicated in bold. * indicates signi\ufb01cant\ndifference at level 0.05 with respect to the InferSent baseline.\n\nModel\n\nMR\n\nCR\n\nSUBJ MPQA SST2\n\nTREC MRPC\n\nSICK-E SICK-R STS14\n\nInferSent-SNLI-ours\n\nINFERSENTAUTOENC\n\ne-INFERSENT\n\n78.18\n(0.25)\n75.94*\n(0.18)\n77.76\n(0.44)\n\n81.28\n(0.15)\n79.26*\n(0.36)\n81.3\n(0.16)\n\n92.46\n(0.15)\n91.72*\n(0.28)\n92.14*\n(0.21)\n\n88.46\n(0.21)\n88.16\n(0.26)\n88.78*\n(0.22)\n\n82.12\n(0.22)\n80.9*\n(0.48)\n81.84\n(0.4)\n\n89.32\n(0.5)\n90.52*\n(0.52)\n90\n(0.51)\n\n74.82 / 82.74\n(0.66 / 0.27)\n76.2* / 82.48\n(0.93 / 1.23)\n75.56 / 83.24*\n(0.62 / 0.24)\n\n85.96\n(0.32)\n85.58\n(0.33)\n85.92\n(0.52)\n\n0.887\n(0.002)\n0.88*\n(0)\n0.89*\n(0)\n\n0.65 / 0.63\n(0 / 0)\n0.5* / 0.5*\n(0.02 / 0.02)\n0.68 / 0.65*\n(0.01 / 0.01)\n\nWe notice that INFERSENTAUTOENC is performing signi\ufb01cantly worse than InferSent on 6 tasks and\nsigni\ufb01cantly outperforms this baseline on only 2 tasks. This indicates that just adding a language\ngenerator can harm performance. Instead, e-INFERSENT signi\ufb01cantly outperforms InferSent on 4\ntasks, while it is signi\ufb01cantly outperformed only on 1 task. Therefore, we conclude that training with\nexplanations helps the model to learn overall better sentence representations.\n\n4.5 TRANSFER: Transfer without \ufb01ne-tuning to out-of-domain NLI\n\nTransfer without \ufb01ne-tuning to out-of-domain entailment datasets is known to exhibit poor perfor-\nmance. For example, Bowman et al. [3] obtained an accuracy of only 46.7% when training on SNLI\nand evaluating on SICK-E [20]. We test how our explanations affect the direct transfer in both label\nprediction and explanation generation by looking at SICK-E [20] and MultiNLI [26]. The latter\nincludes a diverse range of genres of written and spoken English, as well as test sets for cross-genre\ntransfer.\n\nModel We again use our already trained e-INFERSENT model from PREDICTANDEXPLAIN.\n\nResults In Table 4, we present the performance of e-INFERSENT and our 2 baselines when evaluated\nwithout \ufb01ne-tuning on SICK-E and MultiNLI. We notice that the accuracy improvements obtained\nwith e-INFERSENT are very small. However, e-INFERSENT additionally provides explanations,\nwhich could bring insight into the inner workings of the model. We manually annotated the \ufb01rst 100\nexplanations of the test sets. The percentage of correct explanations in the subset where the label was\npredicted correctly was 30.64% for SICK-E and only 1.92% for MultiNLI. We also noticed that the\nexplanations in SICK-E, even when wrong, were generally on-topic and valid statements, while the\nones in MultiNLI were generally nonsense or off-topic. Therefore, transfer learning for generating\nexplanations in out-of-domain NLI would constitute challenging future work.\n\n5 Related work\n\nTable 4: The average performance over 5 seeds of\ne-INFERSENT and the 2 baselines on SICK-E and\nMultiNLI with no \ufb01ne-tuning. Standard deviations\nare in parenthesis.\n\nInterpretability One main direction in inter-\npretability for neural networks is providing ex-\ntractive justi\ufb01cations, i.e., explanations consist-\ning of subsets of the raw input, such as words\nor image patches. Extractive techniques can be\ndivided into post-hoc (applied after training) and\narchitecture-incorporated (guiding the training).\nFor example, Ribeiro et al. [24] introduce a post-\nhoc extractive technique, LIME, that explains\nthe prediction of any classi\ufb01er via a local linear\napproximation around the prediction. Alvarez-\nMelis and Jaakkola [1] introduces a similar ap-\nproach but for structured prediction, where a variational autoencoder provides relevant perturbations\nof the inputs that are then used to infer pairs of input-output tokens that are causally related. While\nthese models provide valuable insight for detecting biases, further model and dataset re\ufb01nements\nwould have to be made on a case-by-case basis. For example, Gururangan et al. [13] identi\ufb01ed a set\nof biases in SNLI, but noted that their attempts to remove them would give rise to other biases.\n\nInferSent-SNLI-ours\nINFERSENTAUTOENC\ne-INFERSENT\n\nSICK-E\n\nMultiNLI\n\n53.27 (1.65)\n52.9 (1.77)\n53.54 (1.43)\n\n57 (0.41)\n55.38 (0.9)\n57.16 (0.51)\n\nModel\n\n8\n\n\fAttention-based models, such as [2, 25], offer some degree of interpretability and have been shown\nto also improve performance on downstream tasks. However, soft attention, the most prominent\nattention model, often does not learn to single out human-interpretable inputs.\n\nNeither extractive nor attention-based techniques can provide full-sentence explanations of a model\u2019s\ndecisions. Moreover, they cannot capture \ufb01ne-grained relations and asymmetries, especially in a task\nlike recognizing textual entailment. For example, if the words person, woman, mountain, outdoors\nare extracted as justi\ufb01cation, one may not know whether the model correctly learned that A woman is\na person and not that A person is a woman, let alone that the model correctly paired (woman, person)\nand (mountain, outside).\n\nNatural language explanations In our work, we have taken a step further and built a neural network\nthat is able to directly provide full-sentence natural language justi\ufb01cations. There has been little\nwork on incorporating and outputting natural language free-form explanations, mostly due to the\nlack of appropriate datasets. In this direction, and very similar to our approach, is the recent work by\nPark et al. [23], who introduce two datasets of natural language explanations for the tasks of visual\nquestion-answering and activity recognition. Another work in this direction is that of Ling et al. [18],\nwho introduced a dataset of textual justi\ufb01cations for solving math problems and formulate the task in\nterms of program execution. Nonetheless, their setup is speci\ufb01c to the task of solving math problems,\nand thus hard to transfer to more general natural understanding tasks. Jansen et al. [16] provided a\ndataset of natural language explanation graphs for elementary science questions. However, with only\n1, 680 pairs of questions and explanations, their corpus is orders of magnitude smaller than e-SNLI.\n\nBreaking natural language inference Recently, an increasing amount of analysis has been carried\nout on the SNLI dataset and on the inner workings of different models trained on it. For example,\nDasgupta et al. [9] assembled a dataset to test whether inference models actually capture composition-\nality beyond word level. They showed that InferSent sentence embeddings [7] indeed do not exhibit\nsigni\ufb01cant compositionality and that downstream models using these sentence representations largely\nrely on simple heuristics that are ecologically valid in the SNLI corpus. For example, high overlap\nin words between premise and hypothesis usually predicts entailment, while most contradictory\nsentence pairs have no or very little overlap of words. Negation words would also strongly indicate\na contradiction. Glockner et al. [11] introduce a toy dataset, BreakingNLI, to test whether natural\nlanguage inference models capture world knowledge and generalize beyond statistical regularities. To\nconstruct BreakingSNLI, they modi\ufb01ed some of the original SNLI sentences such that they differ by\nat most one word from the sentences in the training set. Glockner et al. show that models achieving\nhigh accuracies on SNLI, such as [21, 22, 6], show dramatically reduced performance on this simpler\ndataset, while the model of Chen et al. [5] is more robust due to incorporating external knowledge.\nAs the explanations in e-SNLI are mostly self-contained, our dataset provides the precise external\nknowledge that one requires in order to solve the SNLI inference task. It is therefore a perfect testbed\nfor developing models that incorporate external knowledge from free-form natural language.\n\n6 Conclusions and future work\n\nWe introduced e-SNLI, a large dataset of natural language explanations for an in\ufb02uential task\nof recognizing textual entailment. To demonstrate the usefulness of e-SNLI, we experimented\nwith various ways of using these explanations for outputting human-interpretable full-sentence\njusti\ufb01cations of classi\ufb01cation decisions. We also investigated the usefulness of these explanations as\nan additional training signal for learning better universal sentence representations and the transfer\ncapabilities to out-of-domain NLI datasets. In this work, we established a series of baselines using\nstraight-forward recurrent neural network architectures for incorporating and generating natural\nlanguage explanations. We hope that e-SNLI will be valuable for future research on more advanced\nmodels that would outperform our baselines.\n\nFinally, we hope that the community will explore the dataset in other directions. For example, we\nalso recorded the highlighted words, which we release with the dataset. Similar to the evaluation\nperformed for visual question answering in Das et al. [8], our highlighted words could provide a\nsource of supervision and evaluation for attention models [25, 22] or post-hoc explanation models\nwhere the explanation consists of a subset of the input.\n\n9\n\n\fAcknowledgments This work was supported by the Alan Turing Institute under the EPSRC grant\nEP/N510129/1. We would also like to thank Jakob Foerster for the valuable discussions.\n\nReferences\n\n[1] Alvarez-Melis, D. and Jaakkola, T. S. (2017). A causal framework for explaining the predictions\n\nof black-box sequence-to-sequence models. CoRR, abs/1707.01943.\n\n[2] Bahdanau, D., Cho, K., and Bengio, Y. (2014). Neural machine translation by jointly learning to\n\nalign and translate. CoRR, abs/1409.0473.\n\n[3] Bowman, S. R., Angeli, G., Potts, C., and Manning, C. D. (2015). A large annotated corpus for\n\nlearning natural language inference. CoRR, abs/1508.05326.\n\n[4] Chan, W., Jaitly, N., Le, Q. V., and Vinyals, O. (2015). Listen, attend and spell. CoRR,\n\nabs/1508.01211.\n\n[5] Chen, Q., Zhu, X., Ling, Z., Inkpen, D., and Wei, S. (2017). Natural language inference with\n\nexternal knowledge. CoRR, abs/1711.04289.\n\n[6] Chen, Q., Zhu, X., Ling, Z., Wei, S., and Jiang, H. (2016). Enhancing and combining sequential\n\nand tree LSTM for natural language inference. CoRR, abs/1609.06038.\n\n[7] Conneau, A., Kiela, D., Schwenk, H., Barrault, L., and Bordes, A. (2017). Supervised learning of\nuniversal sentence representations from natural language inference data. CoRR, abs/1705.02364.\n\n[8] Das, A., Agrawal, H., Zitnick, C. L., Parikh, D., and Batra, D. (2016). Human attention in\nvisual question answering: Do humans and deep networks look at the same regions? CoRR,\nabs/1606.03556.\n\n[9] Dasgupta, I., Guo, D., Stuhlm\u00fcller, A., Gershman, S. J., and Goodman, N. D. (2018). Evaluating\n\ncompositionality in sentence embeddings.\n\n[10] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. (2009). ImageNet: A large-scale\n\nhierarchical image database. In Proc. of CVPR.\n\n[11] Glockner, M., Shwartz, V., and Goldberg, Y. (2018). Breaking NLI systems with sentences that\n\nrequire simple lexical inferences. In Proc. of ACL.\n\n[12] Gong, Y., Luo, H., and Zhang, J. (2017). Natural language inference over interaction space.\n\nCoRR, abs/1709.04348.\n\n[13] Gururangan, S., Swayamdipta, S., Levy, O., Schwartz, R., Bowman, S., and Smith, N. A. (2018).\n\nAnnotation artifacts in natural language inference data. In Proc. of NAACL.\n\n[14] Hill, F., Cho, K., and Korhonen, A. (2016). Learning distributed representations of sentences\n\nfrom unlabelled data. CoRR, abs/1602.03483.\n\n[15] Hochreiter, S. and Schmidhuber, J. (1997). Long short-term memory. Neural Comput.,\n\n9(8):1735\u20131780.\n\n[16] Jansen, P. A., Wainwright, E., Marmorstein, S., and Morrison, C. T. (2018). Worldtree: A\ncorpus of explanation graphs for elementary science questions supporting multi-hop inference.\nCoRR, abs/1802.03052.\n\n[17] Kiros, R., Zhu, Y., Salakhutdinov, R., Zemel, R. S., Torralba, A., Urtasun, R., and Fidler, S.\n\n(2015). Skip-thought vectors. CoRR, abs/1506.06726.\n\n[18] Ling, W., Yogatama, D., Dyer, C., and Blunsom, P. (2017). Program induction by rationale\n\ngeneration: Learning to solve and explain algebraic word problems. CoRR, abs/1705.04146.\n\n[19] Liu, P., Qiu, X., and Huang, X. (2016). Modelling interaction of sentence pair with coupled-\n\nlstms. CoRR, abs/1605.05573.\n\n10\n\n\f[20] Marelli, M., Menini, S., Baroni, M., Bentivogli, L., bernardi, R., and Zamparelli, R. (2014). A\n\nsick cure for the evaluation of compositional distributional semantic models.\n\n[21] Nie, Y. and Bansal, M. (2017). Shortcut-stacked sentence encoders for multi-domain inference.\nIn Proceedings of the 2nd Workshop on Evaluating Vector Space Representations for NLP, pages\n41\u201345. Association for Computational Linguistics.\n\n[22] Parikh, A. P., T\u00e4ckstr\u00f6m, O., Das, D., and Uszkoreit, J. (2016). A decomposable attention\n\nmodel for natural language inference. CoRR, abs/1606.01933.\n\n[23] Park, D. H., Hendricks, L. A., Akata, Z., Rohrbach, A., Schiele, B., Darrell, T., and Rohrbach,\nM. (2018). Multimodal explanations: Justifying decisions and pointing to the evidence. CoRR,\nabs/1802.08129.\n\n[24] Ribeiro, M. T., Singh, S., and Guestrin, C. (2016). \"why should I trust you?\": Explaining the\n\npredictions of any classi\ufb01er. CoRR, abs/1602.04938.\n\n[25] Rockt\u00e4schel, T., Grefenstette, E., Hermann, K. M., Kocisk\u00fd, T., and Blunsom, P. (2015).\n\nReasoning about entailment with neural attention. CoRR, abs/1509.06664.\n\n[26] Williams, A., Nangia, N., and Bowman, S. R. (2017). A broad-coverage challenge corpus for\n\nsentence understanding through inference. CoRR, abs/1704.05426.\n\n[27] Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A. C., Salakhutdinov, R., Zemel, R. S., and Bengio,\nY. (2015). Show, attend and tell: Neural image caption generation with visual attention. CoRR,\nabs/1502.03044.\n\nAppendices\n\nA List of templates to \ufb01lter uninformative explanations\n\nGeneral templates\n\n\"<premise>\"\n\n\"<hypothesis>\"\n\n\"<hypothesis> <premise>\"\n\n\"<premise> <hypothesis>\"\n\n\"Sentence 1 states <premise>. Sentence 2 is stating <hypothesis>\"\n\n\"Sentence 2 states <hypothesis>. Sentence 1 is stating <premise>\"\n\n\"There is <hypothesis>\"\n\n\"There is <premise>\"\n\nEntailment templates\n\n\"<premise> implies <hypothesis>\"\n\n\"If <premise> then <hypothesis>\"\n\n\"<premise> would imply <hypothesis>\"\n\n\"<hypothesis> is a rephrasing of <premise>\"\n\n\"<premise> is a rephrasing of <hypothesis>\"\n\n\"In both sentences <hypothesis>\"\n\n\"<premise> would be <hypothesis>\"\n\n\"<premise> can also be said as <hypothesis>\"\n\n11\n\n\f", "award": [], "sourceid": 5810, "authors": [{"given_name": "Oana-Maria", "family_name": "Camburu", "institution": "University of Oxford"}, {"given_name": "Tim", "family_name": "Rockt\u00e4schel", "institution": "University of Oxford"}, {"given_name": "Thomas", "family_name": "Lukasiewicz", "institution": "University of Oxford"}, {"given_name": "Phil", "family_name": "Blunsom", "institution": "Oxford University"}]}