{"title": "End-to-End Learning on 3D Protein Structure for Interface Prediction", "book": "Advances in Neural Information Processing Systems", "page_first": 15642, "page_last": 15651, "abstract": "Despite an explosion in the number of experimentally determined, atomically detailed structures of biomolecules, many critical tasks in structural biology remain data-limited.  Whether performance in such tasks can be improved by using large repositories of tangentially related structural data remains an open question.  To address this question, we focused on a central problem in biology: predicting how proteins interact with one another\u2014that is, which surfaces of one protein bind to those of another protein.  We built a training dataset, the Database of Interacting Protein Structures (DIPS), that contains biases but is two orders of magnitude larger than those used previously.  We found that these biases significantly degrade the performance of existing methods on gold-standard data.  Hypothesizing that assumptions baked into the hand-crafted features on which these methods depend were the source of the problem, we developed the first end-to-end learning model for protein interface prediction, the Siamese Atomic Surfacelet Network (SASNet).  Using only spatial coordinates and identities of atoms, SASNet outperforms state-of-the-art methods trained on gold-standard structural data, even when trained on only 3% of our new dataset.  Code and data available at https://github.com/drorlab/DIPS.", "full_text": "End-to-End Learning on\n\n3D Protein Structure for Interface Prediction\n\nRaphael J. L. Townshend\n\nStanford University\n\nraphael@cs.stanford.edu\n\nPatricia A. Suriana\nStanford University\n\npsuriana@stanford.edu\n\nRishi Bedi\n\nStanford University\n\nrbedi@cs.stanford.edu\n\nRon O. Dror\n\nStanford University\n\nrondror@cs.stanford.edu\n\nAbstract\n\nDespite an explosion in the number of experimentally determined, atomically de-\ntailed structures of biomolecules, many critical tasks in structural biology remain\ndata-limited. Whether performance in such tasks can be improved by using large\nrepositories of tangentially related structural data remains an open question. To\naddress this question, we focused on a central problem in biology: predicting how\nproteins interact with one another\u2014that is, which surfaces of one protein bind to\nthose of another protein. We built a training dataset, the Database of Interacting\nProtein Structures (DIPS), that contains biases but is two orders of magnitude larger\nthan those used previously. We found that these biases signi\ufb01cantly degrade the\nperformance of existing methods on gold-standard data. Hypothesizing that assump-\ntions baked into the hand-crafted features on which these methods depend were the\nsource of the problem, we developed the \ufb01rst end-to-end learning model for protein\ninterface prediction, the Siamese Atomic Surfacelet Network (SASNet). Using only\nspatial coordinates and identities of atoms, SASNet outperforms state-of-the-art\nmethods trained on gold-standard structural data, even when trained on only 3% of\nour new dataset. Code and data available at https://github.com/drorlab/DIPS.\n\n1\n\nIntroduction\n\nProteins are large molecules responsible for executing almost every cellular process. Their function\ndepends critically on their ability to bind to one another in speci\ufb01c ways, forming larger machines\nknown as protein complexes. In this work we tackle the problem of paired protein interface prediction:\ngiven the separate structures of two proteins, we wish to predict which surfaces of the two proteins\nwill come into contact upon binding. This is in contrast to the single-interface prediction problem,\nwhere one wishes to predict which parts of a single protein are likely to form interfaces. Correctly\npredicting protein interfaces has important applications in protein engineering and drug development.\nA large number of experimental structures of protein complexes are available, but\u2014as in many struc-\ntural biology tasks\u2014the amount of supervised data available for paired protein interface prediction\nremains limited. Few gold-standard cases exist in which structures are available both for two proteins\nbound to one another and for each of the two proteins on its own. We wondered if much larger sets of\nstructural data might be deployed in service of tasks such as protein interface prediction.\nTo investigate this problem, we mine the Protein Data Bank (PDB) [1] to construct a large dataset\nof protein complex structures for which structures of the individual proteins on their own are not\navailable. We introduce the Database of Interacting Protein Structures (DIPS), which comprises\n42,826 binary protein interactions\u2014an increase of more than two orders of magnitude over previously\n\n33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada.\n\n\fFigure 1: Protein Binding. The BNI1 protein (blue) opens up to bind to actin (red). While our method\nis trained only using structures of complexes such as the one at right, without any information on\nhow the individual proteins deformed upon binding, we test on pairs of unbound structures such as\nthose at left with minimal loss in performance.\n\nused datasets such as Docking Benchmark 5 (DB5). However, we \ufb01nd that existing state-of-the-art\nmethods are unable to effectively leverage this larger dataset, likely because the assumptions built\ninto these methods\u2019 hand-crafted features are not robust to differences between this training data and\nthe gold-standard test data set.\nWe therefore present SASNet, the \ufb01rst end-to-end learning method applied to interface prediction.\nInstead of relying on hand-engineered, high-level features, we work directly at the atomic level, using\nonly atom positions and identities as inputs. To predict whether an amino acid on the surface of\none protein interacts with an amino acid on the surface of another protein, we voxelize the local\natomic environments, or \"surfacelets,\" surrounding each of them and then apply a siamese-like\nthree-dimensional convolutional neural network to the resulting grids. SASNet outperforms existing\nmethods for structure-based interface prediction while leaving the door open to substantially greater\nperformance improvements not available to competing methods, as we have so far trained on less\nthan 3% of DIPS (due to computational limitations), whereas standard models are already using all\nof the gold-standard training data available to them.\nThere is good reason to believe that convolutional neural networks would be an appropriate \ufb01t for this\nproblem and others in structural biology. For one, the available data is homogeneous in its underlying\nrepresentation: we are given a collection of atoms a \u2208 A where A = P \u00d7 E such that P = R3 is\nthe position space and E = {C, N, O, S, ...} is the set of possible atom element types. We are also\nespecially interested in modeling proximal interactions due to the local nature of the underlying\nphysical forces, a natural strength of the convolutional \ufb01lters. Finally, the stacked nature of neural\nnetworks approximates the hierarchical nature of biomolecular structure: for example a protein can\nbe progressively broken down into domains, secondary structure elements (e.g. alpha helices and\nbeta sheets), amino acids, and \ufb01nally atoms.\nHowever, a major surprise relates to the primary source of bias in DIPS: the proteins within are\nprovided only in their \ufb01nal bound form, in which their shapes almost always match perfectly with\none another. This is in sharp contrast to real tests cases such as those included in DB5, in which the\nstructures of the individual proteins typically lack shape complementarity because proteins tend to\ndeform substantially upon binding. Even though SASNet does not explicitly account for the fact that\nproteins deform upon binding, when we train on DIPS and test on DB5, our method outperforms\nstate-of-the-art techniques that exploit hand-engineered features and are trained directly on DB5.\nThis performance even in the face of such signi\ufb01cant bias is especially exciting as the set of possible\ncon\ufb01gurations a protein can take on when bound to a partner is a subset of all its possible con\ufb01gura-\ntions. Protein interfaces must take on a speci\ufb01c con\ufb01guration upon binding in order to \ufb01t together\nin an energetically favorable manner (i.e., the atoms are more restricted to particular positions; see\nFigure 1) [2]. DIPS only contains proteins in conformations that can already \ufb01t together, whereas\nDB5 also contains protein conformations that require major deformations before being able to \ufb01t\ntogether. Our model\u2019s ability to perform well on DB5 indicates the model has not simply memorized\nthe rules governing interaction in our DIPS dataset (e.g., by searching for shape complementarity).\nInstead, it has learned a representation that at least partially encodes the \ufb02exibility of proteins, without\nbeing explicitly trained to do so, unlike previously reported methods.\n\n2\n\n\f2 Related Work\n\nThere has been signi\ufb01cant interest in applying machine learning methods to biomolecules such as\nproteins, DNA, RNA, and small drug-like molecules. Graph-based approaches have been used for\nderiving properties of small molecules [3], [4], [5], such as predicting the results of quantum mechan-\nical calculations. You et al. [6] employed graph policy networks to generate new molecules. Another\ncommon representation for quantum mechanical calculations is based on Behler and Parrinello [7]\u2019s\nsymmetry functions which use manually determined Gaussian basis functions, as in [8], [9]. Gomes\net al. [10] uses the symmetry functions for protein-ligand binding af\ufb01nity prediction. Equivariant\nnetworks represent another recent and exciting line of work extending these symmetry functions\n[11], [12], [13]. 3D convolutional networks have been used for protein-ligand binding af\ufb01nity [14],\n[15], [16], as well as for protein fold quality assessment [17], protein structure classication [18],\n\ufb01ngerprint prediction [19], and \ufb01lling in missing amino acids [20]. [21] use variational autoencoders\nto create coarse grain molecular dynamics simulations. [22] develop topology-based networks to\npredict biomolecular properties. These tasks differ substantially from protein interface prediction,\nhowever, in that they are much less data-limited.\nTurning to the problem of paired interface prediction, methods developed by Fout et al. [23] and\nSanchez-Garcia et al. [24] have the highest reported performance. They both apply machine learning\ntechniques (graph convolutions and extreme gradient boosting, respectively) to hand-designed se-\nquence conservation and structural features and are trained only on DB5. They choose to represent\nthe protein at the amino acid level, and their structural features capture coarse-grained structure.\nThese features include, for example, a measure of exposure of each amino acid to solvent and the\nnumber of other amino acids in a half-sphere oriented along an amino acid\u2019s side chain. These\nfeatures do not, however, capture more detailed information such as the geometric arrangement of\natoms in an amino acid\u2019s side chains. For the distinct task of single interface prediction, also known\nas binding site prediction, methods such as Jordan et al. [25], Porollo and Meller [26], Northey et al.\n[27], and Hwang et al. [28] also use high-level structural features to predict interfacial residues, but\nin a non-partner-speci\ufb01c manner\u2014given a single protein, these methods predict which of its amino\nacids are likely to form an interface with any other protein. We choose to focus on paired interface\nprediction as Ahmad and Mizuguchi [29] demonstrated that partner-speci\ufb01c interface predictors\nyield much higher performance. Paired interface prediction is also of importance to protein\u2013protein\ndocking, the computational task of predicting the three-dimensional structure of a complex from\nits individual proteins. Docking software currently achieves low accuracy [30]: the lack of robust\ninterface predictors for ranking candidate complexes has been identi\ufb01ed as one of the primary issues\npreventing better performance [31].\nSequences of related proteins (e.g,. sequence conservation and coevolution) represent another source\nof information for addressing the interface prediction problem. The basic idea is that interfacial\nsurfaces of a protein are typically constrained in how they can evolve, as too much variability can\ninterrupt interactions that might be vital to protein function. For example, Ahmad and Mizuguchi\n[29] uses neural networks trained on such features. Given that all these interfaces are determined\nby the physics of actual three-dimensional interactions, the relegation of structure to a hidden and\nunmodeled variable leads to limitations of these approaches. The general consensus in the \ufb01eld is that\nthe performance of purely sequence-based methods is approaching their limit [32]. Information about\nrelated proteins, including using known protein interactions as templates, can boost the performance\nof structure-based methods, but here we study the problem of how best to predict the interface\nbetween two proteins given only the structures of the two proteins \u2014 both because we wish to focus\non identifying optimal structural features and because information about related proteins is not always\navailable, particularly for designed or engineered proteins.\nOur contributions to the problem of paired interface prediction include the \ufb01rst use of end-to-end\nlearning, as well as learned structural features that achieve state-of-the-art performance. Furthermore,\nwe mine the novel DIPS dataset and demonstrate that end-to-end learning instead of hand-engineering\nfeatures enables us to leverage these sorts of much larger structural biology datasets\u2014despite their\ninherent biases.\n\n3\n\n\fDataset\nDB5\nDIPS\n\n# Binary Complexes\n\n# Amino Acid Interactions\n\n230\n\n42,826\n\n21,091\n\n5,767,093\n\nTable 1: Dataset Sizes. By training on complexes from the newly created DIPS dataset, as opposed\nto restricting ourselves to complexes with unbound data available such as those from DB5, we can\naccess over two orders of magnitude more training data than would otherwise be available.\n\n3 Dataset\n\nThe best existing methods for protein interface prediction rely on the Docking Benchmark 5 (DB5)\ndataset [30]. This gold-standard set contains most known labeled examples for the protein interface\nprediction problem. It is also relatively small: 230 complexes in total. Interfacial amino acids (i.e., the\nlabels) are de\ufb01ned based on the structure of the two proteins bound together, but the three-dimensional\nstructures used as input to the model are those of the two proteins when they are unbound. The data\ndistribution therefore closely matches that which we would see when predicting interfaces for new\nexamples, which are provided in their unbound states as we do not know the structure of the resulting\ncomplex a priori. Additionally, the range of dif\ufb01culties and of interaction types in this dataset (e.g.,\nenzyme-inhibitor, antibody-antigen) provides good coverage of typical test cases one might see in\nthe wild. State-of-the-art methods [23], [24] further split DB5 into a training/validation set of 175\ncomplexes, DB5-train, corresponding to DB4 (the complexes from the previous version, Docking\nBenchmark 4) and a test set, DB5-test, of 55 complexes (the complexes added in the update from\nDB4 to DB5). This time-based split simulates the ability of these methods to predict unreleased\ncomplexes, as opposed to a random split which has more training/testing cross-contamination. For\ncomparison we also use DB5-test as our test set.\nWhile DB5 includes only 230 complexes, the PDB contains over 160,000 structures, providing an\nalluring target for increasing the amount of training data available. We therefore set out to construct\nthe Database of Interacting Protein Structures (DIPS) by mining the PDB for pairs of interacting\nproteins (Figure 2A). For this dataset, both the input structures to the model and the labels (that is,\nwhether or not a given amino acid in a \ufb01rst protein physically contacts a given amino acid in the\nother protein) are derived from the structure of the complex in which the two proteins are bound\ntogether. As the PDB contains data of varying quality, we only include complexes that meet the\nfollowing criteria: \u2265 500 \u00c52 buried surface area, solved using X-ray crystallography or cryo-electron\nmicroscopy at better than 3.5 \u00c5 resolution, only contains protein chains longer than 50 amino acids,\nand is the \ufb01rst model in a structure. As DB5 is also derived from the PDB we use sequence-based\npruning to ensure that there is no cross-contamination between our train and test sets. Speci\ufb01cally, we\nexclude any complex that has any individual protein with over 30% sequence identity when aligned\nto any protein in DB5. This is a commonly used sequence identity threshold [33], [25], but competing\nmethods for protein interface prediction do not employ such pruning on their training set, which may\nbias performance comparisons in their favor. The initial processing as well as the sequence-level\nexclusion yields a dataset of 42,826 binary complexes, over two orders of magnitude larger than DB5.\nFor both of these datasets, once these binary protein complexes are generated, we identify all\ninteracting pairs of amino acids. A pair of amino acids \u2014 one from each protein \u2014 is determined\nto be interacting if any of their non-hydrogen atoms (hydrogen atoms are typically not observed in\nexperimental structures) are within 6 \u00c5 of one another (Figure 2B) (as also used by [23], [24]). This\nleads to a total of over \ufb01ve million pairs labelled as positives in DIPS (Figure 2C, see Table 1 for\nexact counts). For the negatives, at train and validation we select random pairs of non-interacting\namino acids spanning the same protein complexes, ensuring a \ufb01xed ratio of positives to negatives\nfrom each complex (Figure 2D, the exact ratio being de\ufb01ned by hyperparameter search, see Section\n4). At test time we use all pairs, to match real-world conditions.\nAs noted previously, the distribution of structures in DIPS differs from that in DB5. For example,\npre-bound proteins in DIPS have a much higher degree of shape complementarity than those in DB5,\nas the former exclusively comprises pairs that are in the correct conformation to bind with one another.\nWe thus must carefully consider our model design so that we can effectively leverage this much larger\nset to tackle the problem of paired protein interface prediction.\n\n4\n\n\fFigure 2: Protein Interface Prediction via SASNet. We predict which parts of two proteins will\ninteract by constructing a binary classi\ufb01er. To extract training examples for the problem, we start\nwith a pair of proteins in complex sampled from DIPS (A, proteins shown in cartoon form), and from\nthere extract all pairs of interacting amino acids (B, atoms shown in stick form). We then split these\npairs to obtain our positives (C), with all remaining non-interacting pairs forming our negatives (D,\nnegatives are down-sampled at train time, but not at test time). These pairs are then individually\nvoxelized into 4D grids, the last dimension being the one-hot encoding of the atom\u2019s element type (E,\natom channel shown as color). These pairs of voxelized representations are then fed through a 3D\nsiamese-like CNN (F, the weights across the two arms are tied).\n\n4 Method\n\nDue to the homogeneous, local, and hierarchical structure of proteins, we selected a three-dimensional\nconvolutional neural network as SASNet\u2019s underlying model (Figure 2F). We \ufb01rst focus on how to\nrepresent our pairs of amino acids and their surrounding environments in order to provide them to our\nnetwork. For each amino acid in a protein, we encode all atoms of that protein within a box centered\non the alpha-carbon of that amino acid \u2014 a region of 3D space that we call a \"surfacelet.\" This\nencodes all structural data local to this central alpha-carbon that is provided in a given PDB structure.\nTo create a dense, three-dimensional, and \ufb01xed-size representation of the input, we choose to voxelize\nthis space (Figure 2D). For each surfacelet, we lay down a grid centered on the alpha carbon of the\namino acid, and record in each voxel the presence or absence of a given atom. A fourth dimension\nis used to encode the element type of the atom, using 4 channels for carbon, oxygen, nitrogen, and\nsulfur, the most commonly found atoms in protein structure (note that hydrogens are typically not\nresolved in experimental structures). In order to build in a notion of rotational invariance, each\ntraining example is randomly rotated, every time it is seen, across the 3 axes of rotation. At test time,\nwe perform 20 random rotations for each example and average the predictions.\nWe feed the voxelized surfacelets to multiple layers of 3D convolution (Conv3D) followed by batch\nnormalization (BN) and recti\ufb01ed linear units (ReLUs), and optionally layers of 3D max pooling\n(MaxPool). We then apply several fully connected (FC) layers followed by more BNs and ReLUs.\nAs we are working with pairs of surfacelets, we employ two such networks with tied weights to build\na latent representation of the two surfacelets, and then concatenate the results. This is a siamese-like\nnetwork, but an important difference from classical siamese approaches, as introduced by Bromley\net al. [34], arises from the nature of the task at hand. Unlike a classical siamese network, we are\nnot attempting to compute a similarity between two objects. This can be shown by considering\nthe nature of protein interactions: a positively charged protein surface is likely to interact with a\nnegatively charged counterpart, even though the two could be considered very dissimilar. Instead of\n\n5\n\n\fMethod\nNGF [4]\nDTNN [35]\nNode+Edge Average [23]\nOrder Dependent [23]\nNode Average [23]\nBIPSPI [24]\nSASNet\n\nCAUROC\n\n0.843 (0.851 +/- 0.010)\n0.861 (0.861 +/- 0.004)\n0.844 (0.850 +/- 0.004)\n0.857 (0.864 +/- 0.006)\n0.876 (0.877 +/- 0.005)\n0.878 (0.878 +/- 0.003)\n0.892 (0.885 +/- 0.009)\n\nTable 2: DB5-test CAUROC performance. For each method we report the CAUROC of the best\nreplicate (as selected by DIPS validation loss for SASNet, and DB5-train loss for others) as well\nas mean and standard deviation of CAUROC across training seeds (see section 5.1). We note that\nwhile competing methods have used all available training data, due to computational limitations\nour SASNet model is trained on less than 3% of our dataset, suggesting an opportunity for further\nperformance improvements.\n\nminimizing Euclidean distance between the two latent representation as would be done in a classical\nsiamese network, we append a series of fully connected layers on the concatenation of the two latent\nrepresentations and optimize the binary cross entropy loss with respect to the original training labels.\nTo determine the optimal model, we ran a large set of manual hyperparameter searches on a limited\nsubset of the full DIPS dataset, created based on selection criteria from [36], randomly sampling a\ntraining and validation set. We varied the dataset size, number of \ufb01lters, number of convolutional\nlayers, number of dense layers, ratio of class imbalance, grid size, grid resolution, and use of max\npooling, batch normalization, and dropout, and selected our models based on average performance\nacross different training seeds on a randomly selected and held out set of DIPS. Surprisingly, most of\nthese parameters had little effect on the overall validation performance, with the notable exception of\nthe positive effect of increasing the overall grid size. Approximately 500 evaluation runs, each with 3\nto 5 different training seeds, were computed in total.\nOur model with the best validation performance involved training on 163840 examples, featurizing a\ngrid of edge length 41 \u00c5 with voxel resolution of 1 \u00c5 (thus starting at a cube size of 41x41x41), and\nthen applying 6 layers of convolution (each of size 3x3x3, with the 6 layers having 32, 32, 64, 64, 128,\n128 convolutional \ufb01lters, respectively) and 2 layers of max pooling, as shown in Figure 2F. A fully\nconnected layer with 512 parameters lays at the top of each tower, and the outputs of both towers are\nconcatenated and passed through two more fully connected layers with 512 parameters each, leading\nto the \ufb01nal prediction. The number of \ufb01lters used in each convolutional layer is doubled every other\nlayer to allow for an increase of the speci\ufb01city of the \ufb01lters as the spatial resolution decreases. We\nuse the RMSProp optimizer with a learning rate of 0.0001. The positive-negative class imbalance\nwas set to 1:1. The overall network is designed such that the grid feeding into the \ufb01rst dense layer is\nsmall enough to avoid memory issues yet large enough to capture important structural information.\nAll models were trained across 4 Titan X GPUs using data-level parallelism, and the best model took\n12 hours to train.\n\n5 Experiments\n\nTo investigate the utility of the additional structural data provided in DIPS, we compare SASNet\u2019s\nperformance to state-of-the-art methods. Furthermore, we demonstrate that competing methods\ntrained on the larger DIPS data set see their DB5 performance severely reduced. Finally, we examine\nthe effect of various model hyperparameters, noting that there is potential for further performance\nimprovements via scaling to a larger fraction of the training dataset. All reported models were run\nacross 3 to 5 training seeds.\nIn our performance comparisons, we utilize only information derivable from the individual protein\nstructures provided, rather than information on evolutionarily related proteins \u2014 both because our\ngoal is to identify the best possible structural features and because information on related proteins is\nnot always available (see Section 2). In particular, we exclude sequence conservation and co-evolution\nfeatures, and re-run the training procedures of the compared models when necessary. We note that, in\n\n6\n\n\fMethod\nNode Average [23]\nBIPSPI [24]\nSASNet\n\nDB5 Trained\n\nDIPS Trained\n\n0.876 (0.877 +/- 0.005)\n0.878 (0.878 +/- 0.003)\n0.876 (0.864 +/- 0.037)\n\n0.712 (0.714 +/- 0.022)\n0.836 (0.836 +/- 0.001)\n0.892 (0.885 +/- 0.009)\n\nTable 3: DB5-test CAUROC for leading methods trained on DB5-train and DIPS. Competing methods\nwith hand-engineered features experience a large drop in performance when trained on DIPS, despite\nits greater size. This indicates the assumptions embedded in their high-level features are not suited to\nthe DIPS dataset. SASNet, on the other hand, increases in performance when trained on DIPS.\n\nthe real world, the interaction between two proteins is determined entirely by the structures of those\ntwo proteins, so the problem we address is a solvable one.\n\n5.1 Comparison to Existing Paired Interface Prediction Methods\n\nWe start by evaluating the effectiveness of our features by comparing to top existing methods applied\nto interface prediction, as shown in Table 2. Graph convolutional network methods based on high-\nlevel features were pulled from the comparison in Fout et al. [23] and include Deep Tensor Neural\nNetworks (DTNN) from Sch\u00fctt et al. [35] and Neural Graph Fingerprints (NGF) from Duvenaud\net al. [4]. Another state-of-the-art feature-engineering method is BIPSPI [24], which is based on\nextreme gradient boosting.\nFor each model, we select from available hyperparameters by choosing those with the best perfor-\nmance on a \ufb01xed data set, across replicates. For SASNet, this set is the validation subset of DIPS,\nwhereas for the other methods this is DB5-train. At test time we evaluate on DB5-test, splitting\nthe predictions by complex and computing the Area Under the Receiver Operating Characteristic\n(AUROC) for each one. We then calculate the median of those AUROCs. We refer to this as the\nmedian per-Complex AUROC (CAUROC). This ensures that larger complexes do not have an outsize\neffect on performance metrics. As our \ufb01nal performance metric we report the CAUROC of the\nreplicate with the best validation performance. SASNet demonstrates superior performance without\nthe use of any hand-engineered features, and without even directly training or validating on DB5.\n\n5.2 Existing Methods Underperform with DIPS\n\nA natural question to ask is whether SASNet\u2019s performance gains are due to the use of the larger\nDIPS dataset for training. If the distribution of bound and unbound were overly similar, then it would\nbe relatively straightforward to leverage the larger size to improve performance. To investigate this,\nwe take state-of-the-art classi\ufb01ers trained on DB5 and instead train them on the same 3% of DIPS we\ntrained SASNet on. We run this procedure on the two competing methods with the highest performing\nstructural features, BIPSPI [24] and Node Average [23].\nInstead of staying even or increasing, the performance of competing methods degrades when trained\non DIPS as opposed to DB5 (Table 3). This re\ufb02ects a lack of robustness to the biases inherent to DIPS.\nOur method, on the other hand, is robust to the use of DIPS for training, allowing us to use the larger\ntraining dataset successfully. We also observe that SASNet trained on DB5 suffers some degradation\nin performance due to the smaller dataset, but remains competitive with the state-of-the-art.\n\n5.3 Hyperparameter Effects\n\nGiven the expense of running 3D convolutions, our best models are limited to being trained on a\nfraction of the full DIPS dataset. We are additionally limited by the size and resolution of the grids\ndue to the cubic relationship between edge size and the total number of voxels. As these are problems\nthat are surmountable through additional engineering effort and compute power, we are interested\nin assessing the potential bene\ufb01ts of scaling up along these axes. We run \ufb01ve training seeds per\ncondition and plot the average and standard deviation of CAUROC across replicates.\nFigure 3A shows the results of the grid size scaling tests, with resolution held \ufb01xed at 1 \u00c5 and total\nnumber of voxels allowed to vary (e.g., grid edge size of 19 would correspond to 19x19x19 voxels).\n\n7\n\n\f(A) Grid size tests, dataset size \ufb01xed to 81920.\n\n(B) Dataset size tests, grid size \ufb01xed to 23 \u00c5.\n\nFigure 3: SASNet bene\ufb01ts from large input sizes (A), and has potential for being further scaled (B).\nWe plot the DB5-test CAUROC mean and standard deviation over \ufb01ve different training seeds.\n\nWe notice consistent performance improvements up to a grid edge size of 27 \u00c5, with performance\nincreases becoming noisier and mostly tapering off afterwards. In Figure 3B, we see that larger dataset\nsize yields consistently increasing performance, indicating that our model is capable of leveraging\nadditional data to increase its performance and achieve state-of-the-art results.\nFinally, as over\ufb01tting is always a danger with high-capacity models, we investigate even more\nstringent exclusion criteria, though these factors are not considered by the state-of-the-art methods\nto whose performance we compare. Rost [37] shows there can still be similarity between structures\nwith sequence identities as low as 20%. Filtering out training examples with 20% or greater sequence\nidentity to any sequence in DB5 does not signi\ufb01cantly impact model performance, resulting in\na CAUROC performance of 0.887. We also investigate structural-level pruning, removing any\ncomplexes in DIPS that share domain-domain interactions with DB5, as de\ufb01ned in Mosca et al. [38].\nSuch pruning does not signi\ufb01cantly affect performance; SASNet still achieves 0.883 CAUROC.\n\n6 Conclusion\n\nIn this work we introduce DIPS, a dataset for interface prediction two orders of magnitude larger\nthan those used previously. As existing methods\u2019 hand-crafted features are unable to cope adequately\nwith the bias present in this dataset, we create SASNet, the \ufb01rst end-to-end learning framework for\ninterface prediction. We surpass current state-of-the-art results on the paired interface prediction\nproblem while only training on proteins already in their bound con\ufb01gurations, without using any\nfeatures identi\ufb01ed by human experts. This is particularly intriguing as proteins are \ufb02exible structures\nthat typically deform at multiple scales upon binding, and DIPS does not capture this deformation.\nThe high performance on DB5 indicates our model has learned complex features beyond simple\nshape complementarity and has captured some notion of protein \ufb02exibility. Furthermore, the small\nnumber of assumptions made combined with the generalizability of the learned features is also of\ninterest, as we can envision improving solutions to many data-poor structural biology problems (such\nas protein design and drug discovery) through training on larger, tangentially related datasets.\nOne hypothesis as to why SASNet\u2019s CNNs are able to generalize so well for this task is that proteins\nform hierarchical structures whose formation is driven primarily by local interatomic forces, making\nprotein structures a good \ufb01t for the stacked convolutional framework. Though these properties are\nwell understood at the lowest levels (only 22 amino acids are genetically encoded, each having a \ufb01xed\natomic composition), the de\ufb01nitions become less precise as we move up the hierarchy. Amino acids\noften form secondary structure elements such as alpha-helices and beta-sheets. At a higher level,\nparts of the protein can form into independent and stable pieces of 3D structure known as protein\ndomains. Many motifs are shared between proteins at all levels of this hierarchy. CNNs may be able\nnot only to capture the known relationships between structural elements at different scales, but also\nto derive new relations that have not been fully characterized. Further investigation of the learned\n\ufb01lters could yield insight into the nature of these higher-level structural patterns, allowing for a better\nunderstanding of protein structure and its relationship to protein-protein interactions.\n\n8\n\n\fAcknowledgments\n\nThe authors thank Guy Amdur, Robin Betz, Stephan Eismann, Scott Hollingsworth, Milind Jagota,\nYianni Laloudakis, Naomi Latorraca, Joe Paggi, Reid Pryzant, Jo\u00e3o Rodrigues, and AJ Venkatakr-\nishnan for their discussions and advice. This work was supported by Intel, Amazon, the National\nScience Foundation Graduate Research Fellowship Program under Grant No. 1147470, the U.S.\nDepartment of Energy Of\ufb01ce of Science Graduate Student Research (SCGSR) program, and the\nU.S. Department of Energy, Of\ufb01ce of Science, Of\ufb01ce of Advanced Scienti\ufb01c Computing Research,\nScienti\ufb01c Discovery through Advanced Computing (SciDAC) program.\n\nReferences\n\n[1] H. M. Berman, J. Westbrook, Z. Feng, G. Gilliland, T. N. Bhat, H. Weissig, I. N. Shindyalov,\nand P. E. Bourne, \u201cThe Protein Data Bank.,\u201d Nucleic Acids Research, vol. 28, no. 1, pp. 235\u201342,\n2000.\n\n[2] D. Kuroda and J. J. Gray, \u201cPushing the Backbone in Protein-Protein Docking,\u201d Structure,\n\nvol. 24, no. 10, pp. 1821\u20131829, 2016.\n\n[3] S. Kearnes, K. McCloskey, M. Berndl, V. Pande, and P. Riley, \u201cMolecular graph convolutions:\nmoving beyond \ufb01ngerprints,\u201d Journal of Computer-Aided Molecular Design, vol. 30, no. 8,\npp. 595\u2013608, 2016.\n\n[5]\n\n[4] D. Duvenaud, D. Maclaurin, J. Aguilera-Iparraguirre, R. G\u00f3mez-Bombarelli, T. Hirzel, A.\nAspuru-Guzik, and R. P. Adams, \u201cConvolutional Networks on Graphs for Learning Molecular\nFingerprints,\u201d in NeurIPS, 2015.\nJ. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl, \u201cNeural Message Passing\nfor Quantum Chemistry,\u201d in International Conference on Machine Learning, 2017, pp. 1263\u2013\n1272. arXiv: 1704.01212.\nJ. You, B. Liu, R. Ying, V. Pande, and J. Leskovec, \u201cGraph Convolutional Policy Network for\nGoal-Directed Molecular Graph Generation,\u201d in NeurIPS, 2018. arXiv: 1806.02473.\nJ. Behler and M. Parrinello, \u201cGeneralized neural-network representation of high-dimensional\npotential-energy surfaces.,\u201d Physical Review Letters, vol. 98, no. 14, p. 146 401, 2007.\n\n[6]\n\n[7]\n\n[9]\n\n[8] K. T. Sch\u00fctt, P.-J. Kindermans, H. E. Sauceda, S. Chmiela, A. Tkatchenko, and K.-R. M\u00fcller,\n\u201cSchNet: A continuous-\ufb01lter convolutional neural network for modeling quantum interactions,\u201d\nJournal of Chemical Theory and Computation, vol. 13, no. 11, pp. 5255\u20135264, 2017. arXiv:\n1706.08566.\nJ. S. Smith, O. Isayev, and A. E. Roitberg, \u201cANI-1: an extensible neural network potential with\nDFT accuracy at force \ufb01eld computational cost,\u201d Chemical Science, vol. 8, no. 4, pp. 3192\u2013\n3203, 2017.\nJ. Gomes, B. Ramsundar, E. N. Feinberg, and V. S. Pande, \u201cAtomic Convolutional Networks\nfor Predicting Protein-Ligand Binding Af\ufb01nity,\u201d 2017. arXiv: 1703.10603.\n\n[10]\n\n[11] N. Thomas, T. Smidt, S. Kearnes, L. Yang, L. Li, K. Kohlhoff, and P. Riley, \u201cTensor Field\nNetworks: Rotation- and Translation-Equivariant Neural Networks for 3D Point Clouds,\u201d 2018.\narXiv: 1802.08219.\n\n[12] R. Kondor, \u201cN-body Networks: a Covariant Hierarchical Neural Network Architecture for\n\nLearning Atomic Potentials,\u201d 2018. arXiv: 1803.01588.\n\n[13] M. Weiler, M. Geiger, M. Welling, W. Boomsma, and T. Cohen, \u201c3D Steerable CNNs: Learning\nRotationally Equivariant Features in Volumetric Data,\u201d in NeurIPS, 2018. arXiv: 1807.02547.\n[14] M. Ragoza, J. Hochuli, E. Idrobo, J. Sunseri, and D. R. Koes, \u201cProtein\u2013Ligand Scoring with\nConvolutional Neural Networks,\u201d Journal of Chemical Information and Modeling, vol. 57,\nno. 4, pp. 942\u2013957, 2017. arXiv: 1612.02751.\nI. Wallach, M. Dzamba, and A. Heifets, \u201cAtomNet: A Deep Convolutional Neural Network for\nBioactivity Prediction in Structure-based Drug Discovery,\u201d 2015. arXiv: 1510.02855.\nJ. Jim\u00e9nez, S. Doerr, G. Mart\u00ednez-Rosell, A. S. Rose, and G. De Fabritiis, \u201cDeepSite: Protein-\nbinding site predictor using 3D-convolutional neural networks,\u201d Bioinformatics, vol. 33, no. 19,\npp. 3036\u20133042, 2017.\n\n[15]\n\n[16]\n\n[17] G. Derevyanko, S. Grudinin, Y. Bengio, and G. Lamoureux, \u201cDeep convolutional networks\nfor quality assessment of protein folds,\u201d Bioinformatics, vol. 34, no. 23, pp. 4046\u20134053, 2018.\narXiv: 1801.06252.\n\n9\n\n\f[18] D. R. de Jesus, J. Cuevas, W. Rivera, and S. Crivelli, \u201cCapsule Networks for Protein Structure\n\nClassi\ufb01cation and Prediction,\u201d 2018. arXiv: 1808.07475.\n\n[19] D. Kuzminykh, D. Polykovskiy, A. Kadurin, A. Zhebrak, I. Baskov, S. Nikolenko, R.\nShayakhmetov, and A. Zhavoronkov, \u201c3D Molecular Representations Based on the Wave\nTransform for Convolutional Neural Networks,\u201d Molecular Pharmaceutics, vol. 15, no. 10,\npp. 4378\u20134385, 2018. arXiv: acs.molpharmaceut.7b01134 [10.1021].\n\n[20] W. Torng and R. B. Altman, \u201c3D deep convolutional neural networks for amino acid environ-\n\nment similarity analysis,\u201d BMC Bioinformatics, vol. 18, no. 1, pp. 1\u201323, 2017.\n\n[21] W. Wang and R. G\u00f3mez-Bombarelli, \u201cVariational Coarse-Graining for Molecular Dynamics,\u201d\n\nTech. Rep., 2018. arXiv: 1812.02706v1.\n\n[22] Z. Cang and G.-W. Wei, \u201cTopologyNet: Topology based deep convolutional neural networks\n\nfor biomolecular property predictions,\u201d 2017. arXiv: 1704.00063.\n\n[23] A. Fout, J. Byrd, B. Shariat, and A. Ben-Hur, \u201cInterface Prediction using Graph Convolutional\n\nNetworks,\u201d in NeurIPS, 2017.\n\n[24] R. Sanchez-Garcia, C. O. S. Sorzano, J. M. Carazo, and J. Segura, \u201cBIPSPI: a method for\nthe prediction of partner-speci\ufb01c protein\u2013protein interfaces,\u201d Bioinformatics, vol. 35, no. 14,\npp. 343\u2013353,\n\n[25] R. A. Jordan, Y. EL-Manzalawy, D. Dobbs, and V. Honavar, \u201cPredicting protein-protein\ninterface residues using local surface structural similarity,\u201d BMC Bioinformatics, vol. 13,\nno. 41, 2012.\n\n[26] A. Porollo and J. Meller, \u201cPrediction-based \ufb01ngerprints of protein-protein interactions,\u201d Pro-\nteins: Structure, Function, and Bioinformatics, vol. 66, no. 3, pp. 630\u2013645, 2006. arXiv:\n0605018 [q-bio].\n\n[27] T. C. Northey, A. Bare\u0161i\u00b4c, and A. C. R. Martin, \u201cIntPred: a structure-based predictor of\nprotein\u2013protein interaction sites,\u201d Bioinformatics, vol. 34, no. 2, A. Valencia, Ed., pp. 223\u2013229,\n2018.\n\n[28] H. Hwang, D. Petrey, and B. Honig, \u201cA Hybrid Method for Protein-Protein Interface Predic-\n\ntion,\u201d Protein Science, vol. 25, no. 1, pp. 159\u2013165, 2016.\n\n[29] S. Ahmad and K. Mizuguchi, \u201cPartner-aware prediction of interacting residues in protein-\n\nprotein complexes from sequence data,\u201d PLOS ONE, vol. 6, no. 12, 2011.\n\n[30] T. Vreven, I. H. Moal, A. Vangone, B. G. Pierce, P. L. Kastritis, M. Torchala, R. Chaleil,\nB. Jim\u00e9nez-Garc\u00eda, P. A. Bates, J. Fernandez-Recio, A. M. Bonvin, and Z. Weng, \u201cUpdates to\nthe Integrated Protein\u2013Protein Interaction Benchmarks: Docking Benchmark Version 5 and\nAf\ufb01nity Benchmark Version 2,\u201d Journal of Molecular Biology, vol. 427, no. 19, pp. 3031\u20133041,\n2015.\n\n[31] A. M. Bonvin, \u201cFlexible protein-protein docking,\u201d Current Opinion in Structural Biology,\n\nvol. 16, no. 2, pp. 194\u2013200, 2006.\n\n[33]\n\n[32] R. Esmaielbeiki, K. Krawczyk, B. Knapp, J.-C. Nebel, and C. M. Deane, \u201cProgress and\nchallenges in predicting protein interfaces,\u201d Brie\ufb01ngs in Bioinformatics, vol. 17, no. 1, pp. 117\u2013\n131, 2016.\nJ. Yang, A. Roy, and Y. Zhang, \u201cProtein\u2013ligand binding site recognition using complementary\nbinding-speci\ufb01c substructure comparison and sequence pro\ufb01le alignment,\u201d Bioinformatics,\nvol. 29, no. 20, pp. 2588\u20132595, 2013.\nJ. Bromley, J. W. Bentz, L. Bottou, I. Guyon, Y. Lecun, C. Moore, E. S\u00e4ckinger, and R. Shah,\n\u201cSignature Veri\ufb01cation using a \"Siamese\" Time Delay Neural Network,\u201d NeurIPS, pp. 737\u2013744,\n1993. arXiv: 1406.1078.\n\n[34]\n\n[35] K. T. Sch\u00fctt, F. Arbabzadah, S. Chmiela, K. R. M\u00fcller, and A. Tkatchenko, \u201cQuantum-chemical\ninsights from deep tensor neural networks,\u201d Nature Communications, vol. 8, p. 13 890, 2017.\n[36] T. Kirys, A. M. Ruvinsky, D. Singla, A. V. Tuzikov, P. J. Kundrotas, and I. A. Vakser, \u201cSimu-\nlated unbound structures for benchmarking of protein docking in the Dockground resource,\u201d\nBMC Bioinformatics, vol. 16, no. 1, p. 243, 2015.\n\n[37] B. Rost, \u201cTwilight zone of protein sequence alignments,\u201d Protein engineering, vol. 12, no. 2,\n\npp. 85\u201394, 1999.\n\n[38] R. Mosca, A. C\u00e9ol, A. Stein, R. Olivella, and P. Aloy, \u201c3did: A catalog of domain-based\ninteractions of known three-dimensional structure,\u201d Nucleic Acids Research, vol. 42, no. D1,\npp. 374\u2013379, 2014.\n\n10\n\n\f", "award": [], "sourceid": 9078, "authors": [{"given_name": "Raphael", "family_name": "Townshend", "institution": "Stanford University"}, {"given_name": "Rishi", "family_name": "Bedi", "institution": "System1 Biosciences"}, {"given_name": "Patricia", "family_name": "Suriana", "institution": "Stanford University"}, {"given_name": "Ron", "family_name": "Dror", "institution": "Stanford University"}]}