{"title": "Learning Perceptual Inference by Contrasting", "book": "Advances in Neural Information Processing Systems", "page_first": 1075, "page_last": 1087, "abstract": "\u201cThinking in pictures,\u201d [1] i.e., spatial-temporal reasoning, effortless and instantaneous for humans, is believed to be a significant ability to perform logical induction and a crucial factor in the intellectual history of technology development. Modern Artificial Intelligence (AI), fueled by massive datasets, deeper models, and mighty computation, has come to a stage where (super-)human-level performances are observed in certain specific tasks. However, current AI's ability in \u201cthinking in pictures\u201d is still far lacking behind. In this work, we study how to improve machines' reasoning ability on one challenging task of this kind: Raven's Progressive Matrices (RPM). Specifically, we borrow the very idea of \u201ccontrast effects\u201d from the field of psychology, cognition, and education to design and train a permutation-invariant model. Inspired by cognitive studies, we equip our model with a simple inference module that is jointly trained with the perception backbone. Combining all the elements, we propose the Contrastive Perceptual Inference network (CoPINet) and empirically demonstrate that CoPINet sets the new state-of-the-art for permutation-invariant models on two major datasets. We conclude that spatial-temporal reasoning depends on envisaging the possibilities consistent with the relations between objects and can be solved from pixel-level inputs.", "full_text": "Learning Perceptual Inference by Contrasting\n\nChi Zhang(cid:63),1,4, Baoxiong Jia(cid:63),1, Feng Gao3,4, Yixin Zhu3,4, Hongjing Lu2, Song-Chun Zhu1,3,4\n\n1 Department of Computer Science, University of California, Los Angeles\n\n2 Department of Psychology, University of California, Los Angeles\n3 Department of Statistics, University of California, Los Angeles\n\n4 International Center for AI and Robot Autonomy (CARA)\n\n{chi.zhang,baoxiongjia,f.gao,yixin.zhu,hongjing,sczhu}@ucla.edu\n\nAbstract\n\n\u201cThinking in pictures,\u201d [1] i.e., spatial-temporal reasoning, effortless and instanta-\nneous for humans, is believed to be a signi\ufb01cant ability to perform logical induction\nand a crucial factor in the intellectual history of technology development. Modern\nArti\ufb01cial Intelligence (AI), fueled by massive datasets, deeper models, and mighty\ncomputation, has come to a stage where (super-)human-level performances are\nobserved in certain speci\ufb01c tasks. However, current AI\u2019s ability in \u201cthinking in\npictures\u201d is still far lacking behind. In this work, we study how to improve ma-\nchines\u2019 reasoning ability on one challenging task of this kind: Raven\u2019s Progressive\nMatrices (RPM). Speci\ufb01cally, we borrow the very idea of \u201ccontrast effects\u201d from\nthe \ufb01eld of psychology, cognition, and education to design and train a permutation-\ninvariant model. Inspired by cognitive studies, we equip our model with a simple\ninference module that is jointly trained with the perception backbone. Combin-\ning all the elements, we propose the Contrastive Perceptual Inference network\n(CoPINet) and empirically demonstrate that CoPINet sets the new state-of-the-art\nfor permutation-invariant models on two major datasets. We conclude that spatial-\ntemporal reasoning depends on envisaging the possibilities consistent with the\nrelations between objects and can be solved from pixel-level inputs.\n\n1\n\nIntroduction\n\nAmong the broad spectrum of computer vision tasks are ones where dramatic progress has been\nwitnessed, especially those involving visual information retrieval [2\u20135]. Signi\ufb01cant improvement\nhas also manifested itself in tasks associating visual and linguistic understanding [6\u20139]. However, it\nwas only until recently that the research community started to re-investigate tasks relying heavily\non the ability of \u201cthinking in pictures\u201d with modern AI approaches [1, 10, 11], particularly spatial-\ntemporal inductive reasoning [12\u201314]; this line of work primarily focuses on Raven\u2019s Progressive\nMatrices (RPM) [15, 16]. It is believed that RPM is closely related to real intelligence [17], diagnostic\nof abstract and structural reasoning ability [18], and characterizes \ufb02uid intelligence [19\u201322]. In such\na test, subjects are provided with two rows of \ufb01gures following certain unknown rules and asked\nto pick the correct answer from the choices that would best complete the third row with a missing\nentry; see Figure 1(a) for an example. As shown in early works [12, 14], despite the fact that visual\nelements are relatively straightforward, there is still a notable performance gap between human and\nmachine visual reasoning in this challenging task.\nOne missing ingredient that may result in this performance gap is a proper form of contrasting\nmechanism. Originated from perceptual learning [23, 24], it is well established in the \ufb01eld of\npsychology and education [25\u201329] that teaching new concepts by comparing with noisy examples is\n\n(cid:63) indicates equal contribution.\n\n33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada.\n\n\fquite effective. Smith and Gentner [30] summarize that comparing cases facilitates transfer learning\nand problem-solving, as well as the ability to learn relational categories. Gentner [31] in his structure-\nmapping theory points out that learners generate a structure alignment between two representation\nwhen they compare two cases. A more recent study from Schwartz et al. [32] also shows that\ncontrasting cases help foster an appreciation of a deep understanding of concepts.\nWe argue that such a contrast effect [33], found in both humans and animals [34\u201338], is essential to\nmachines\u2019 reasoning ability as well. With access to how the data is generated, a recent attempt [13]\n\ufb01nds that models demonstrate better generalizability if the choice of data and the manner in which\nit is presented to the model are made \u201ccontrastive.\u201d In this paper, we try to address a more direct\nand challenging question, independent of how the data is generated: how to incorporate an explicit\ncontrasting mechanism during model training in order to improve machines\u2019 reasoning ability?\nSpeci\ufb01cally, we come up with two levels of contrast in our model: a novel contrast module and\na new contrast loss. At the model level, we design a permutation-invariant contrast module that\nsummarizes the common features and distinguishes each candidate by projecting it onto its residual on\nthe common feature space. At the objective level, we leverage ideas in contrastive estimation [39\u201341]\nand propose a variant of Noise-Contrastive Estimation (NCE) loss.\nAnother reason why RPM is challenging for existing machine reasoning systems could be attributed\nto the demanding nature of the interplay between perception and inference. Carpenter et al. [17]\npostulate that a proper understanding of one RPM instance requires not only an accurate encoding\nof individual elements and their visual attributes but also the correct induction of the hidden rules.\nIn other words, to solve RPM, machine reasoning systems are expected to be equipped with both\nperception and inference subsystems; lacking either component would only result in a sub-optimal\nsolution. While existing work primarily focuses on perception, we propose to bridge this gap with\na simple inference module jointly trained with the perception backbone; speci\ufb01cally, the inference\nmodule reasons about which category the current problem instance falls into. Instead of training the\ninference module to predict the ground-truth category, we borrow the basis learning idea from [42]\nand jointly learn the inference subsystem with perception. This basis formulation could also be\nregarded as a hidden variable and trained using a log probability estimate.\nFurthermore, we hope to make a critical improvement to the model design such that it is truly\npermutation-invariant. The invariance is mandatory, as an ideal RPM solver should not change the\nrepresentation simply because the rows or columns of answer candidates are swapped or the order\nof the choices alters. This characteristic is an essential trait missed by all recent works [12, 14].\nSpeci\ufb01cally, Zhang et al. [12] stack all choices in the channel dimension and feed it into the network\nin one pass. Barrett et al. [14] add additional positional tagging to their Wild Relational Network\n(WReN). Both of them explicitly make models permutation-sensitive. We notice in our experiments\nthat removing the positional tagging in WReN decreases the performance by 28%, indicating that the\nmodel bypasses the intrinsic complexity of RPM by remembering the positional association. Making\nthe model permutation-invariant also shifts the problem from classi\ufb01cation to ranking.\nCombining contrasting, perceptual inference, and permutation invariance, we propose the Contrastive\nPerceptual Inference network (CoPINet). To verify its effectiveness, we conduct comprehensive\nexperiments on two major datasets: the RAVEN dataset [12] and the PGM dataset [14]. Empirical\nstudies show that our model achieves human-level performance on RAVEN and a new record on\nPGM, setting new state-of-the-art for permutation-invariant models on the two datasets. Further\nablation on RAVEN and PGM reveals how each component contributes to performance improvement.\nWe also investigate how the model performance varies under different sizes of datasets, as a step\ntowards an ideal machine reasoning system capable of low-shot learning.\nThis paper makes four major contributions:\n\u2022 We introduce two levels of contrast to improve machines\u2019 reasoning ability in RPM. At the model\nlevel, we design a contrast module that aggregates common features and projects each candidate\nto its residual. At the objective level, we use an NCE loss variant instead of the cross-entropy to\nencourage contrast effects.\n\nanswer candidates, shifting the previous view of RPM from classi\ufb01cation to ranking.\n\nbackbone jointly. Instead of using ground-truth, we regularize it with a \ufb01xed number of bases.\n\n\u2022 Inspired by Carpenter et al. [17], we incorporate an inference module to learn with the perception\n\u2022 We make our model permutation-invariant in terms of swapped rows or columns and shuf\ufb02ed\n\u2022 Combining ideas above, we propose CoPINet that sets new state-of-the-art on two major datasets.\n\n2\n\n\fFigure 1: (a) An example of RPM. The hidden rule(s) in this problem can be denoted as {[OR, line, type]},\nwhere an OR operation is applied to the type attribute of all lines, following the notations in Barrett et al. [14].\nIt is further noted that the OR operation is applied row-wise, and there is only one choice that satis\ufb01es the\nrow-wise OR constraint. Hence the correct answer should be 5. (b) The proposed CoPINet architecture. Given a\nRPM problem, the inference branch samples a most likely rule for each attribute based only on the context O of\nthe problem. Sampled rules are transformed and fed into each contrast module in the perception branch. Note\nthat the combination of the contrast module and the residual block can be repeated. Dashed lines indicate that\nparameters are shared among the modules. (c) A sketch of the contrast module.\n\n2 Related Work\n\nContrastive Learning Teaching concepts by comparing cases, or contrasting, has proven effective\nin both human learning and machine learning. Gentner [31] postulates that human\u2019s learning-by-\ncomparison process is a structural mapping and alignment process. A later article [43] \ufb01rmly supports\nthis conjecture and shows \ufb01nding the individual difference is easier for humans when similar items\nare compared. Recently, Smith and Gentner [30] conclude that learning by comparing two contrastive\ncases facilitates the distinction between two complex interrelated relational concepts. Evidence\nin educational research further strengthens the importance of contrasting\u2014quantitative structure\nof empirical phenomena is less demanding to learn when contrasting cases are used [32, 44, 45].\nAll the literature calls for a similar treatment of contrast in machine learning. While techniques\nfrom [46\u201348] are based on triplet loss using max margin to separate positive and negative samples,\nnegative contrastive samples and negative sampling are proposed for language modeling [40] and\nword embedding [49, 50], respectively. Gutmann and Hyv\u00e4rinen [39] discuss a general learning\nframework called Noise-Contrastive Estimation (NCE) for estimating parameters by taking noise\nsamples into consideration, which Dai and Lin [41] follow to learn an effective image captioning\nmodel. A recent work [13] leverages contrastive learning in RPM; however, it focuses on data\npresentation while leaving the question of modeling and learning unanswered.\n\nComputational Models on RPM The cognitive science community is the \ufb01rst to investigate RPM\nwith computational models. Assuming access to a perfect state representation, structure-mapping\ntheory [31] and the high-level perception theory of analogy [51, 52] are designed with heuristics to\nsolve the RPM problem at a symbolic level [17, 53\u201355]. Another stream of research approaches the\nproblem by measuring the image similarity with hand-crafted state representations [56\u201360]. More\nrecently, end-to-end data-driven methods with raw image input are proposed [12\u201314, 61]. Wang and\nSu [61] introduce an automatic RPM generation method. Barrett et al. [14] release the \ufb01rst large-scale\nRPM dataset and present a relational model [62] designed for it. Steenbrugge et al. [63] propose a\npretrained \u03b2-VAE to improve the generalization performance of models on RPM. Zhang et al. [12]\nprovide another dataset with structural annotations using stochastic image grammar [64\u201366]. Hill\net al. [13] take a different approach and study how data presentation affects learning.\n\n3 Learning Perceptual Inference by Contrasting\n\nThe task of RPM can be formally de\ufb01ned as: given a list of observed images O = {oi}8\ni=1, forming\na 3 \u00d7 3 matrix with a \ufb01nal missing element, a solver aims to \ufb01nd an answer a(cid:63) from an unordered set\n\n3\n\n(a)AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoMQm3CXRsugjWVE8wHJEfY2e8mSvb1jd04IR36CjYUitv4iO/+Nm+QKTXww8Hhvhpl5QSKFQdf9dgobm1vbO8Xd0t7+weFR+fikbeJUM95isYx1N6CGS6F4CwVK3k00p1EgeSeY3M79zhPXRsTqEacJ9yM6UiIUjKKVHqr0clCuuDV3AbJOvJxUIEdzUP7qD2OWRlwhk9SYnucm6GdUo2CSz0r91PCEsgkd8Z6likbc+Nni1Bm5sMqQhLG2pZAs1N8TGY2MmUaB7Ywojs2qNxf/83ophtd+JlSSIldsuShMJcGYzP8mQ6E5Qzm1hDIt7K2EjammDG06JRuCt/ryOmnXa55b8+7rlcZNHkcRzuAcquDBFTTgDprQAgYjeIZXeHOk8+K8Ox/L1oKTz5zCHzifP4gLjUg=AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoMQm3CXRsugjWVE8wHJEfY2e8mSvb1jd04IR36CjYUitv4iO/+Nm+QKTXww8Hhvhpl5QSKFQdf9dgobm1vbO8Xd0t7+weFR+fikbeJUM95isYx1N6CGS6F4CwVK3k00p1EgeSeY3M79zhPXRsTqEacJ9yM6UiIUjKKVHqr0clCuuDV3AbJOvJxUIEdzUP7qD2OWRlwhk9SYnucm6GdUo2CSz0r91PCEsgkd8Z6likbc+Nni1Bm5sMqQhLG2pZAs1N8TGY2MmUaB7Ywojs2qNxf/83ophtd+JlSSIldsuShMJcGYzP8mQ6E5Qzm1hDIt7K2EjammDG06JRuCt/ryOmnXa55b8+7rlcZNHkcRzuAcquDBFTTgDprQAgYjeIZXeHOk8+K8Ox/L1oKTz5zCHzifP4gLjUg=AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoMQm3CXRsugjWVE8wHJEfY2e8mSvb1jd04IR36CjYUitv4iO/+Nm+QKTXww8Hhvhpl5QSKFQdf9dgobm1vbO8Xd0t7+weFR+fikbeJUM95isYx1N6CGS6F4CwVK3k00p1EgeSeY3M79zhPXRsTqEacJ9yM6UiIUjKKVHqr0clCuuDV3AbJOvJxUIEdzUP7qD2OWRlwhk9SYnucm6GdUo2CSz0r91PCEsgkd8Z6likbc+Nni1Bm5sMqQhLG2pZAs1N8TGY2MmUaB7Ywojs2qNxf/83ophtd+JlSSIldsuShMJcGYzP8mQ6E5Qzm1hDIt7K2EjammDG06JRuCt/ryOmnXa55b8+7rlcZNHkcRzuAcquDBFTTgDprQAgYjeIZXeHOk8+K8Ox/L1oKTz5zCHzifP4gLjUg=AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoMQm3CXRsugjWVE8wHJEfY2e8mSvb1jd04IR36CjYUitv4iO/+Nm+QKTXww8Hhvhpl5QSKFQdf9dgobm1vbO8Xd0t7+weFR+fikbeJUM95isYx1N6CGS6F4CwVK3k00p1EgeSeY3M79zhPXRsTqEacJ9yM6UiIUjKKVHqr0clCuuDV3AbJOvJxUIEdzUP7qD2OWRlwhk9SYnucm6GdUo2CSz0r91PCEsgkd8Z6likbc+Nni1Bm5sMqQhLG2pZAs1N8TGY2MmUaB7Ywojs2qNxf/83ophtd+JlSSIldsuShMJcGYzP8mQ6E5Qzm1hDIt7K2EjammDG06JRuCt/ryOmnXa55b8+7rlcZNHkcRzuAcquDBFTTgDprQAgYjeIZXeHOk8+K8Ox/L1oKTz5zCHzifP4gLjUg=FO[a1AAACC3icbVDLSsNAFJ34rPUVdelmaBFclUQEXRYFcWcF+4AmhJvppB06eTAzEUrI3o2/4saFIm79AXf+jZM2iLYeGDhzzr3ce4+fcCaVZX0ZS8srq2vrlY3q5tb2zq65t9+RcSoIbZOYx6Lng6ScRbStmOK0lwgKoc9p1x9fFn73ngrJ4uhOTRLqhjCMWMAIKC15Zs0JQY0I8Owq97Kfz02OHZImGDw798y61bCmwIvELkkdlWh55qcziEka0kgRDlL2bStRbgZCMcJpXnVSSRMgYxjSvqYRhFS62fSWHB9pZYCDWOgXKTxVf3dkEEo5CX1dWSwr571C/M/rpyo4dzMWJamiEZkNClKOVYyLYPCACUoUn2gCRDC9KyYjEECUjq+qQ7DnT14knZOGbTXs29N686KMo4IOUQ0dIxudoSa6Ri3URgQ9oCf0gl6NR+PZeDPeZ6VLRtlzgP7A+PgGMS6bHA==AAACC3icbVDLSsNAFJ34rPUVdelmaBFclUQEXRYFcWcF+4AmhJvppB06eTAzEUrI3o2/4saFIm79AXf+jZM2iLYeGDhzzr3ce4+fcCaVZX0ZS8srq2vrlY3q5tb2zq65t9+RcSoIbZOYx6Lng6ScRbStmOK0lwgKoc9p1x9fFn73ngrJ4uhOTRLqhjCMWMAIKC15Zs0JQY0I8Owq97Kfz02OHZImGDw798y61bCmwIvELkkdlWh55qcziEka0kgRDlL2bStRbgZCMcJpXnVSSRMgYxjSvqYRhFS62fSWHB9pZYCDWOgXKTxVf3dkEEo5CX1dWSwr571C/M/rpyo4dzMWJamiEZkNClKOVYyLYPCACUoUn2gCRDC9KyYjEECUjq+qQ7DnT14knZOGbTXs29N686KMo4IOUQ0dIxudoSa6Ri3URgQ9oCf0gl6NR+PZeDPeZ6VLRtlzgP7A+PgGMS6bHA==AAACC3icbVDLSsNAFJ34rPUVdelmaBFclUQEXRYFcWcF+4AmhJvppB06eTAzEUrI3o2/4saFIm79AXf+jZM2iLYeGDhzzr3ce4+fcCaVZX0ZS8srq2vrlY3q5tb2zq65t9+RcSoIbZOYx6Lng6ScRbStmOK0lwgKoc9p1x9fFn73ngrJ4uhOTRLqhjCMWMAIKC15Zs0JQY0I8Owq97Kfz02OHZImGDw798y61bCmwIvELkkdlWh55qcziEka0kgRDlL2bStRbgZCMcJpXnVSSRMgYxjSvqYRhFS62fSWHB9pZYCDWOgXKTxVf3dkEEo5CX1dWSwr571C/M/rpyo4dzMWJamiEZkNClKOVYyLYPCACUoUn2gCRDC9KyYjEECUjq+qQ7DnT14knZOGbTXs29N686KMo4IOUQ0dIxudoSa6Ri3URgQ9oCf0gl6NR+PZeDPeZ6VLRtlzgP7A+PgGMS6bHA==AAACC3icbVDLSsNAFJ34rPUVdelmaBFclUQEXRYFcWcF+4AmhJvppB06eTAzEUrI3o2/4saFIm79AXf+jZM2iLYeGDhzzr3ce4+fcCaVZX0ZS8srq2vrlY3q5tb2zq65t9+RcSoIbZOYx6Lng6ScRbStmOK0lwgKoc9p1x9fFn73ngrJ4uhOTRLqhjCMWMAIKC15Zs0JQY0I8Owq97Kfz02OHZImGDw798y61bCmwIvELkkdlWh55qcziEka0kgRDlL2bStRbgZCMcJpXnVSSRMgYxjSvqYRhFS62fSWHB9pZYCDWOgXKTxVf3dkEEo5CX1dWSwr571C/M/rpyo4dzMWJamiEZkNClKOVYyLYPCACUoUn2gCRDC9KyYjEECUjq+qQ7DnT14knZOGbTXs29N686KMo4IOUQ0dIxudoSa6Ri3URgQ9oCf0gl6NR+PZeDPeZ6VLRtlzgP7A+PgGMS6bHA==FO[a2AAACC3icbVDLSsNAFL2pr1pfUZduhhbBVUmKoMuiIO6sYB/QhjCZTtqhkwczE6GE7N34K25cKOLWH3Dn3zhpg2jrgYEz59zLvfd4MWdSWdaXUVpZXVvfKG9WtrZ3dvfM/YOOjBJBaJtEPBI9D0vKWUjbiilOe7GgOPA47XqTy9zv3lMhWRTeqWlMnQCPQuYzgpWWXLM6CLAaE8zTq8xNfz43GRqQJEbYbWSuWbPq1gxomdgFqUGBlmt+DoYRSQIaKsKxlH3bipWTYqEY4TSrDBJJY0wmeET7moY4oNJJZ7dk6FgrQ+RHQr9QoZn6uyPFgZTTwNOV+bJy0cvF/7x+ovxzJ2VhnCgakvkgP+FIRSgPBg2ZoETxqSaYCKZ3RWSMBSZKx1fRIdiLJy+TTqNuW3X79rTWvCjiKMMRVOEEbDiDJlxDC9pA4AGe4AVejUfj2Xgz3uelJaPoOYQ/MD6+ATKzmx0=AAACC3icbVDLSsNAFL2pr1pfUZduhhbBVUmKoMuiIO6sYB/QhjCZTtqhkwczE6GE7N34K25cKOLWH3Dn3zhpg2jrgYEz59zLvfd4MWdSWdaXUVpZXVvfKG9WtrZ3dvfM/YOOjBJBaJtEPBI9D0vKWUjbiilOe7GgOPA47XqTy9zv3lMhWRTeqWlMnQCPQuYzgpWWXLM6CLAaE8zTq8xNfz43GRqQJEbYbWSuWbPq1gxomdgFqUGBlmt+DoYRSQIaKsKxlH3bipWTYqEY4TSrDBJJY0wmeET7moY4oNJJZ7dk6FgrQ+RHQr9QoZn6uyPFgZTTwNOV+bJy0cvF/7x+ovxzJ2VhnCgakvkgP+FIRSgPBg2ZoETxqSaYCKZ3RWSMBSZKx1fRIdiLJy+TTqNuW3X79rTWvCjiKMMRVOEEbDiDJlxDC9pA4AGe4AVejUfj2Xgz3uelJaPoOYQ/MD6+ATKzmx0=AAACC3icbVDLSsNAFL2pr1pfUZduhhbBVUmKoMuiIO6sYB/QhjCZTtqhkwczE6GE7N34K25cKOLWH3Dn3zhpg2jrgYEz59zLvfd4MWdSWdaXUVpZXVvfKG9WtrZ3dvfM/YOOjBJBaJtEPBI9D0vKWUjbiilOe7GgOPA47XqTy9zv3lMhWRTeqWlMnQCPQuYzgpWWXLM6CLAaE8zTq8xNfz43GRqQJEbYbWSuWbPq1gxomdgFqUGBlmt+DoYRSQIaKsKxlH3bipWTYqEY4TSrDBJJY0wmeET7moY4oNJJZ7dk6FgrQ+RHQr9QoZn6uyPFgZTTwNOV+bJy0cvF/7x+ovxzJ2VhnCgakvkgP+FIRSgPBg2ZoETxqSaYCKZ3RWSMBSZKx1fRIdiLJy+TTqNuW3X79rTWvCjiKMMRVOEEbDiDJlxDC9pA4AGe4AVejUfj2Xgz3uelJaPoOYQ/MD6+ATKzmx0=AAACC3icbVDLSsNAFL2pr1pfUZduhhbBVUmKoMuiIO6sYB/QhjCZTtqhkwczE6GE7N34K25cKOLWH3Dn3zhpg2jrgYEz59zLvfd4MWdSWdaXUVpZXVvfKG9WtrZ3dvfM/YOOjBJBaJtEPBI9D0vKWUjbiilOe7GgOPA47XqTy9zv3lMhWRTeqWlMnQCPQuYzgpWWXLM6CLAaE8zTq8xNfz43GRqQJEbYbWSuWbPq1gxomdgFqUGBlmt+DoYRSQIaKsKxlH3bipWTYqEY4TSrDBJJY0wmeET7moY4oNJJZ7dk6FgrQ+RHQr9QoZn6uyPFgZTTwNOV+bJy0cvF/7x+ovxzJ2VhnCgakvkgP+FIRSgPBg2ZoETxqSaYCKZ3RWSMBSZKx1fRIdiLJy+TTqNuW3X79rTWvCjiKMMRVOEEbDiDJlxDC9pA4AGe4AVejUfj2Xgz3uelJaPoOYQ/MD6+ATKzmx0=FO[a8AAACC3icbVDLSsNAFL2pr1pfUZduhhbBVUlEsMuiIO6sYB/QhjCZTtqhkwczE6GE7N34K25cKOLWH3Dn3zhpg2jrgYEz59zLvfd4MWdSWdaXUVpZXVvfKG9WtrZ3dvfM/YOOjBJBaJtEPBI9D0vKWUjbiilOe7GgOPA47XqTy9zv3lMhWRTeqWlMnQCPQuYzgpWWXLM6CLAaE8zTq8xNfz43GRqQJEbYbWSuWbPq1gxomdgFqUGBlmt+DoYRSQIaKsKxlH3bipWTYqEY4TSrDBJJY0wmeET7moY4oNJJZ7dk6FgrQ+RHQr9QoZn6uyPFgZTTwNOV+bJy0cvF/7x+ovyGk7IwThQNyXyQn3CkIpQHg4ZMUKL4VBNMBNO7IjLGAhOl46voEOzFk5dJ57RuW3X79qzWvCjiKMMRVOEEbDiHJlxDC9pA4AGe4AVejUfj2Xgz3uelJaPoOYQ/MD6+ATvRmyM=AAACC3icbVDLSsNAFL2pr1pfUZduhhbBVUlEsMuiIO6sYB/QhjCZTtqhkwczE6GE7N34K25cKOLWH3Dn3zhpg2jrgYEz59zLvfd4MWdSWdaXUVpZXVvfKG9WtrZ3dvfM/YOOjBJBaJtEPBI9D0vKWUjbiilOe7GgOPA47XqTy9zv3lMhWRTeqWlMnQCPQuYzgpWWXLM6CLAaE8zTq8xNfz43GRqQJEbYbWSuWbPq1gxomdgFqUGBlmt+DoYRSQIaKsKxlH3bipWTYqEY4TSrDBJJY0wmeET7moY4oNJJZ7dk6FgrQ+RHQr9QoZn6uyPFgZTTwNOV+bJy0cvF/7x+ovyGk7IwThQNyXyQn3CkIpQHg4ZMUKL4VBNMBNO7IjLGAhOl46voEOzFk5dJ57RuW3X79qzWvCjiKMMRVOEEbDiHJlxDC9pA4AGe4AVejUfj2Xgz3uelJaPoOYQ/MD6+ATvRmyM=AAACC3icbVDLSsNAFL2pr1pfUZduhhbBVUlEsMuiIO6sYB/QhjCZTtqhkwczE6GE7N34K25cKOLWH3Dn3zhpg2jrgYEz59zLvfd4MWdSWdaXUVpZXVvfKG9WtrZ3dvfM/YOOjBJBaJtEPBI9D0vKWUjbiilOe7GgOPA47XqTy9zv3lMhWRTeqWlMnQCPQuYzgpWWXLM6CLAaE8zTq8xNfz43GRqQJEbYbWSuWbPq1gxomdgFqUGBlmt+DoYRSQIaKsKxlH3bipWTYqEY4TSrDBJJY0wmeET7moY4oNJJZ7dk6FgrQ+RHQr9QoZn6uyPFgZTTwNOV+bJy0cvF/7x+ovyGk7IwThQNyXyQn3CkIpQHg4ZMUKL4VBNMBNO7IjLGAhOl46voEOzFk5dJ57RuW3X79qzWvCjiKMMRVOEEbDiHJlxDC9pA4AGe4AVejUfj2Xgz3uelJaPoOYQ/MD6+ATvRmyM=AAACC3icbVDLSsNAFL2pr1pfUZduhhbBVUlEsMuiIO6sYB/QhjCZTtqhkwczE6GE7N34K25cKOLWH3Dn3zhpg2jrgYEz59zLvfd4MWdSWdaXUVpZXVvfKG9WtrZ3dvfM/YOOjBJBaJtEPBI9D0vKWUjbiilOe7GgOPA47XqTy9zv3lMhWRTeqWlMnQCPQuYzgpWWXLM6CLAaE8zTq8xNfz43GRqQJEbYbWSuWbPq1gxomdgFqUGBlmt+DoYRSQIaKsKxlH3bipWTYqEY4TSrDBJJY0wmeET7moY4oNJJZ7dk6FgrQ+RHQr9QoZn6uyPFgZTTwNOV+bJy0cvF/7x+ovyGk7IwThQNyXyQn3CkIpQHg4ZMUKL4VBNMBNO7IjLGAhOl46voEOzFk5dJ57RuW3X79qzWvCjiKMMRVOEEbDiHJlxDC9pA4AGe4AVejUfj2Xgz3uelJaPoOYQ/MD6+ATvRmyM=AAAB+HicbVDLSsNAFL3xWeujUZduBovgqiQi6LLoxmUF+4AmlMl00g6dTMI8hBr6JW5cKOLWT3Hn3zhps9DWAwOHc+7lnjlRxpnSnvftrK1vbG5tV3aqu3v7BzX38KijUiMJbZOUp7IXYUU5E7Stmea0l0mKk4jTbjS5LfzuI5WKpeJBTzMaJngkWMwI1lYauLUgwXocxXmQZtyo2cCtew1vDrRK/JLUoURr4H4Fw5SYhApNOFaq73uZDnMsNSOczqqBUTTDZIJHtG+pwAlVYT4PPkNnVhmiOJX2CY3m6u+NHCdKTZPIThYx1bJXiP95faPj6zBnIjOaCrI4FBuOdIqKFtCQSUo0n1qCiWQ2KyJjLDHRtquqLcFf/vIq6Vw0fK/h31/WmzdlHRU4gVM4Bx+uoAl30II2EDDwDK/w5jw5L86787EYXXPKnWP4A+fzB3Ggk5Y=AAAB+HicbVDLSsNAFL3xWeujUZduBovgqiQi6LLoxmUF+4AmlMl00g6dTMI8hBr6JW5cKOLWT3Hn3zhps9DWAwOHc+7lnjlRxpnSnvftrK1vbG5tV3aqu3v7BzX38KijUiMJbZOUp7IXYUU5E7Stmea0l0mKk4jTbjS5LfzuI5WKpeJBTzMaJngkWMwI1lYauLUgwXocxXmQZtyo2cCtew1vDrRK/JLUoURr4H4Fw5SYhApNOFaq73uZDnMsNSOczqqBUTTDZIJHtG+pwAlVYT4PPkNnVhmiOJX2CY3m6u+NHCdKTZPIThYx1bJXiP95faPj6zBnIjOaCrI4FBuOdIqKFtCQSUo0n1qCiWQ2KyJjLDHRtquqLcFf/vIq6Vw0fK/h31/WmzdlHRU4gVM4Bx+uoAl30II2EDDwDK/w5jw5L86787EYXXPKnWP4A+fzB3Ggk5Y=AAAB+HicbVDLSsNAFL3xWeujUZduBovgqiQi6LLoxmUF+4AmlMl00g6dTMI8hBr6JW5cKOLWT3Hn3zhps9DWAwOHc+7lnjlRxpnSnvftrK1vbG5tV3aqu3v7BzX38KijUiMJbZOUp7IXYUU5E7Stmea0l0mKk4jTbjS5LfzuI5WKpeJBTzMaJngkWMwI1lYauLUgwXocxXmQZtyo2cCtew1vDrRK/JLUoURr4H4Fw5SYhApNOFaq73uZDnMsNSOczqqBUTTDZIJHtG+pwAlVYT4PPkNnVhmiOJX2CY3m6u+NHCdKTZPIThYx1bJXiP95faPj6zBnIjOaCrI4FBuOdIqKFtCQSUo0n1qCiWQ2KyJjLDHRtquqLcFf/vIq6Vw0fK/h31/WmzdlHRU4gVM4Bx+uoAl30II2EDDwDK/w5jw5L86787EYXXPKnWP4A+fzB3Ggk5Y=AAAB+HicbVDLSsNAFL3xWeujUZduBovgqiQi6LLoxmUF+4AmlMl00g6dTMI8hBr6JW5cKOLWT3Hn3zhps9DWAwOHc+7lnjlRxpnSnvftrK1vbG5tV3aqu3v7BzX38KijUiMJbZOUp7IXYUU5E7Stmea0l0mKk4jTbjS5LfzuI5WKpeJBTzMaJngkWMwI1lYauLUgwXocxXmQZtyo2cCtew1vDrRK/JLUoURr4H4Fw5SYhApNOFaq73uZDnMsNSOczqqBUTTDZIJHtG+pwAlVYT4PPkNnVhmiOJX2CY3m6u+NHCdKTZPIThYx1bJXiP95faPj6zBnIjOaCrI4FBuOdIqKFtCQSUo0n1qCiWQ2KyJjLDHRtquqLcFf/vIq6Vw0fK/h31/WmzdlHRU4gVM4Bx+uoAl30II2EDDwDK/w5jw5L86787EYXXPKnWP4A+fzB3Ggk5Y=h(\u00b7)AAAB73icbVBNS8NAEJ3Ur1q/qh69LBahXkoigh6LXjxWsB/QhrLZbNqlm03cnQil9E948aCIV/+ON/+N2zYHbX0w8Hhvhpl5QSqFQdf9dgpr6xubW8Xt0s7u3v5B+fCoZZJMM95kiUx0J6CGS6F4EwVK3kk1p3EgeTsY3c789hPXRiTqAccp92M6UCISjKKVOsNqj4UJnvfLFbfmzkFWiZeTCuRo9MtfvTBhWcwVMkmN6Xpuiv6EahRM8mmplxmeUjaiA961VNGYG38yv3dKzqwSkijRthSSufp7YkJjY8ZxYDtjikOz7M3E/7xuhtG1PxEqzZArtlgUZZJgQmbPk1BozlCOLaFMC3srYUOqKUMbUcmG4C2/vEpaFzXPrXn3l5X6TR5HEU7gFKrgwRXU4Q4a0AQGEp7hFd6cR+fFeXc+Fq0FJ585hj9wPn8AZiSPiQ==AAAB73icbVBNS8NAEJ3Ur1q/qh69LBahXkoigh6LXjxWsB/QhrLZbNqlm03cnQil9E948aCIV/+ON/+N2zYHbX0w8Hhvhpl5QSqFQdf9dgpr6xubW8Xt0s7u3v5B+fCoZZJMM95kiUx0J6CGS6F4EwVK3kk1p3EgeTsY3c789hPXRiTqAccp92M6UCISjKKVOsNqj4UJnvfLFbfmzkFWiZeTCuRo9MtfvTBhWcwVMkmN6Xpuiv6EahRM8mmplxmeUjaiA961VNGYG38yv3dKzqwSkijRthSSufp7YkJjY8ZxYDtjikOz7M3E/7xuhtG1PxEqzZArtlgUZZJgQmbPk1BozlCOLaFMC3srYUOqKUMbUcmG4C2/vEpaFzXPrXn3l5X6TR5HEU7gFKrgwRXU4Q4a0AQGEp7hFd6cR+fFeXc+Fq0FJ585hj9wPn8AZiSPiQ==AAAB73icbVBNS8NAEJ3Ur1q/qh69LBahXkoigh6LXjxWsB/QhrLZbNqlm03cnQil9E948aCIV/+ON/+N2zYHbX0w8Hhvhpl5QSqFQdf9dgpr6xubW8Xt0s7u3v5B+fCoZZJMM95kiUx0J6CGS6F4EwVK3kk1p3EgeTsY3c789hPXRiTqAccp92M6UCISjKKVOsNqj4UJnvfLFbfmzkFWiZeTCuRo9MtfvTBhWcwVMkmN6Xpuiv6EahRM8mmplxmeUjaiA961VNGYG38yv3dKzqwSkijRthSSufp7YkJjY8ZxYDtjikOz7M3E/7xuhtG1PxEqzZArtlgUZZJgQmbPk1BozlCOLaFMC3srYUOqKUMbUcmG4C2/vEpaFzXPrXn3l5X6TR5HEU7gFKrgwRXU4Q4a0AQGEp7hFd6cR+fFeXc+Fq0FJ585hj9wPn8AZiSPiQ==AAAB73icbVBNS8NAEJ3Ur1q/qh69LBahXkoigh6LXjxWsB/QhrLZbNqlm03cnQil9E948aCIV/+ON/+N2zYHbX0w8Hhvhpl5QSqFQdf9dgpr6xubW8Xt0s7u3v5B+fCoZZJMM95kiUx0J6CGS6F4EwVK3kk1p3EgeTsY3c789hPXRiTqAccp92M6UCISjKKVOsNqj4UJnvfLFbfmzkFWiZeTCuRo9MtfvTBhWcwVMkmN6Xpuiv6EahRM8mmplxmeUjaiA961VNGYG38yv3dKzqwSkijRthSSufp7YkJjY8ZxYDtjikOz7M3E/7xuhtG1PxEqzZArtlgUZZJgQmbPk1BozlCOLaFMC3srYUOqKUMbUcmG4C2/vEpaFzXPrXn3l5X6TR5HEU7gFKrgwRXU4Q4a0AQGEp7hFd6cR+fFeXc+Fq0FJ585hj9wPn8AZiSPiQ==...AAAB7XicbVBNS8NAEJ34WetX1aOXYBE8lUQEPRa9eKxgP6ANZbPZtGs3u2F3Uiih/8GLB0W8+n+8+W/ctjlo64OBx3szzMwLU8ENet63s7a+sbm1Xdop7+7tHxxWjo5bRmWasiZVQulOSAwTXLImchSsk2pGklCwdji6m/ntMdOGK/mIk5QFCRlIHnNK0Eqt3jhSaPqVqlfz5nBXiV+QKhRo9CtfvUjRLGESqSDGdH0vxSAnGjkVbFruZYalhI7IgHUtlSRhJsjn107dc6tEbqy0LYnuXP09kZPEmEkS2s6E4NAsezPxP6+bYXwT5FymGTJJF4viTLio3NnrbsQ1oygmlhCqub3VpUOiCUUbUNmG4C+/vEpalzXfq/kPV9X6bRFHCU7hDC7Ah2uowz00oAkUnuAZXuHNUc6L8+58LFrXnGLmBP7A+fwBy2+PQg==AAAB7XicbVBNS8NAEJ34WetX1aOXYBE8lUQEPRa9eKxgP6ANZbPZtGs3u2F3Uiih/8GLB0W8+n+8+W/ctjlo64OBx3szzMwLU8ENet63s7a+sbm1Xdop7+7tHxxWjo5bRmWasiZVQulOSAwTXLImchSsk2pGklCwdji6m/ntMdOGK/mIk5QFCRlIHnNK0Eqt3jhSaPqVqlfz5nBXiV+QKhRo9CtfvUjRLGESqSDGdH0vxSAnGjkVbFruZYalhI7IgHUtlSRhJsjn107dc6tEbqy0LYnuXP09kZPEmEkS2s6E4NAsezPxP6+bYXwT5FymGTJJF4viTLio3NnrbsQ1oygmlhCqub3VpUOiCUUbUNmG4C+/vEpalzXfq/kPV9X6bRFHCU7hDC7Ah2uowz00oAkUnuAZXuHNUc6L8+58LFrXnGLmBP7A+fwBy2+PQg==AAAB7XicbVBNS8NAEJ34WetX1aOXYBE8lUQEPRa9eKxgP6ANZbPZtGs3u2F3Uiih/8GLB0W8+n+8+W/ctjlo64OBx3szzMwLU8ENet63s7a+sbm1Xdop7+7tHxxWjo5bRmWasiZVQulOSAwTXLImchSsk2pGklCwdji6m/ntMdOGK/mIk5QFCRlIHnNK0Eqt3jhSaPqVqlfz5nBXiV+QKhRo9CtfvUjRLGESqSDGdH0vxSAnGjkVbFruZYalhI7IgHUtlSRhJsjn107dc6tEbqy0LYnuXP09kZPEmEkS2s6E4NAsezPxP6+bYXwT5FymGTJJF4viTLio3NnrbsQ1oygmlhCqub3VpUOiCUUbUNmG4C+/vEpalzXfq/kPV9X6bRFHCU7hDC7Ah2uowz00oAkUnuAZXuHNUc6L8+58LFrXnGLmBP7A+fwBy2+PQg==AAAB7XicbVBNS8NAEJ34WetX1aOXYBE8lUQEPRa9eKxgP6ANZbPZtGs3u2F3Uiih/8GLB0W8+n+8+W/ctjlo64OBx3szzMwLU8ENet63s7a+sbm1Xdop7+7tHxxWjo5bRmWasiZVQulOSAwTXLImchSsk2pGklCwdji6m/ntMdOGK/mIk5QFCRlIHnNK0Eqt3jhSaPqVqlfz5nBXiV+QKhRo9CtfvUjRLGESqSDGdH0vxSAnGjkVbFruZYalhI7IgHUtlSRhJsjn107dc6tEbqy0LYnuXP09kZPEmEkS2s6E4NAsezPxP6+bYXwT5FymGTJJF4viTLio3NnrbsQ1oygmlhCqub3VpUOiCUUbUNmG4C+/vEpalzXfq/kPV9X6bRFHCU7hDC7Ah2uowz00oAkUnuAZXuHNUc6L8+58LFrXnGLmBP7A+fwBy2+PQg== AAAB+XicbVDLSgMxFL1TX7W+Rl26CRbBVZkRQZdFNy4r2Ad0hpJJM21oJhmSTKEM/RM3LhRx65+482/MtLPQ1gOBwzn3ck9OlHKmjed9O5WNza3tnepubW//4PDIPT7paJkpQttEcql6EdaUM0HbhhlOe6miOIk47UaT+8LvTqnSTIonM0tpmOCRYDEj2Fhp4LpBgs04ivNAJkxkej5w617DWwCtE78kdSjRGrhfwVCSLKHCEI617vteasIcK8MIp/NakGmaYjLBI9q3VOCE6jBfJJ+jC6sMUSyVfcKghfp7I8eJ1rMkspNFTr3qFeJ/Xj8z8W2YM5FmhgqyPBRnHBmJihrQkClKDJ9ZgoliNisiY6wwMbasmi3BX/3yOulcNXyv4T9e15t3ZR1VOINzuAQfbqAJD9CCNhCYwjO8wpuTOy/Ou/OxHK045c4p/IHz+QM6jZQIAAAB+XicbVDLSgMxFL1TX7W+Rl26CRbBVZkRQZdFNy4r2Ad0hpJJM21oJhmSTKEM/RM3LhRx65+482/MtLPQ1gOBwzn3ck9OlHKmjed9O5WNza3tnepubW//4PDIPT7paJkpQttEcql6EdaUM0HbhhlOe6miOIk47UaT+8LvTqnSTIonM0tpmOCRYDEj2Fhp4LpBgs04ivNAJkxkej5w617DWwCtE78kdSjRGrhfwVCSLKHCEI617vteasIcK8MIp/NakGmaYjLBI9q3VOCE6jBfJJ+jC6sMUSyVfcKghfp7I8eJ1rMkspNFTr3qFeJ/Xj8z8W2YM5FmhgqyPBRnHBmJihrQkClKDJ9ZgoliNisiY6wwMbasmi3BX/3yOulcNXyv4T9e15t3ZR1VOINzuAQfbqAJD9CCNhCYwjO8wpuTOy/Ou/OxHK045c4p/IHz+QM6jZQIAAAB+XicbVDLSgMxFL1TX7W+Rl26CRbBVZkRQZdFNy4r2Ad0hpJJM21oJhmSTKEM/RM3LhRx65+482/MtLPQ1gOBwzn3ck9OlHKmjed9O5WNza3tnepubW//4PDIPT7paJkpQttEcql6EdaUM0HbhhlOe6miOIk47UaT+8LvTqnSTIonM0tpmOCRYDEj2Fhp4LpBgs04ivNAJkxkej5w617DWwCtE78kdSjRGrhfwVCSLKHCEI617vteasIcK8MIp/NakGmaYjLBI9q3VOCE6jBfJJ+jC6sMUSyVfcKghfp7I8eJ1rMkspNFTr3qFeJ/Xj8z8W2YM5FmhgqyPBRnHBmJihrQkClKDJ9ZgoliNisiY6wwMbasmi3BX/3yOulcNXyv4T9e15t3ZR1VOINzuAQfbqAJD9CCNhCYwjO8wpuTOy/Ou/OxHK045c4p/IHz+QM6jZQIAAAB+XicbVDLSgMxFL1TX7W+Rl26CRbBVZkRQZdFNy4r2Ad0hpJJM21oJhmSTKEM/RM3LhRx65+482/MtLPQ1gOBwzn3ck9OlHKmjed9O5WNza3tnepubW//4PDIPT7paJkpQttEcql6EdaUM0HbhhlOe6miOIk47UaT+8LvTqnSTIonM0tpmOCRYDEj2Fhp4LpBgs04ivNAJkxkej5w617DWwCtE78kdSjRGrhfwVCSLKHCEI617vteasIcK8MIp/NakGmaYjLBI9q3VOCE6jBfJJ+jC6sMUSyVfcKghfp7I8eJ1rMkspNFTr3qFeJ/Xj8z8W2YM5FmhgqyPBRnHBmJihrQkClKDJ9ZgoliNisiY6wwMbasmi3BX/3yOulcNXyv4T9e15t3ZR1VOINzuAQfbqAJD9CCNhCYwjO8wpuTOy/Ou/OxHK045c4p/IHz+QM6jZQI AAAB+XicbVDLSgMxFL1TX7W+Rl26CRbBVZkRQZdFNy4r2Ad0hpJJM21oJhmSTKEM/RM3LhRx65+482/MtLPQ1gOBwzn3ck9OlHKmjed9O5WNza3tnepubW//4PDIPT7paJkpQttEcql6EdaUM0HbhhlOe6miOIk47UaT+8LvTqnSTIonM0tpmOCRYDEj2Fhp4LpBgs04ivNAJkxkej5w617DWwCtE78kdSjRGrhfwVCSLKHCEI617vteasIcK8MIp/NakGmaYjLBI9q3VOCE6jBfJJ+jC6sMUSyVfcKghfp7I8eJ1rMkspNFTr3qFeJ/Xj8z8W2YM5FmhgqyPBRnHBmJihrQkClKDJ9ZgoliNisiY6wwMbasmi3BX/3yOulcNXyv4T9e15t3ZR1VOINzuAQfbqAJD9CCNhCYwjO8wpuTOy/Ou/OxHK045c4p/IHz+QM6jZQIAAAB+XicbVDLSgMxFL1TX7W+Rl26CRbBVZkRQZdFNy4r2Ad0hpJJM21oJhmSTKEM/RM3LhRx65+482/MtLPQ1gOBwzn3ck9OlHKmjed9O5WNza3tnepubW//4PDIPT7paJkpQttEcql6EdaUM0HbhhlOe6miOIk47UaT+8LvTqnSTIonM0tpmOCRYDEj2Fhp4LpBgs04ivNAJkxkej5w617DWwCtE78kdSjRGrhfwVCSLKHCEI617vteasIcK8MIp/NakGmaYjLBI9q3VOCE6jBfJJ+jC6sMUSyVfcKghfp7I8eJ1rMkspNFTr3qFeJ/Xj8z8W2YM5FmhgqyPBRnHBmJihrQkClKDJ9ZgoliNisiY6wwMbasmi3BX/3yOulcNXyv4T9e15t3ZR1VOINzuAQfbqAJD9CCNhCYwjO8wpuTOy/Ou/OxHK045c4p/IHz+QM6jZQIAAAB+XicbVDLSgMxFL1TX7W+Rl26CRbBVZkRQZdFNy4r2Ad0hpJJM21oJhmSTKEM/RM3LhRx65+482/MtLPQ1gOBwzn3ck9OlHKmjed9O5WNza3tnepubW//4PDIPT7paJkpQttEcql6EdaUM0HbhhlOe6miOIk47UaT+8LvTqnSTIonM0tpmOCRYDEj2Fhp4LpBgs04ivNAJkxkej5w617DWwCtE78kdSjRGrhfwVCSLKHCEI617vteasIcK8MIp/NakGmaYjLBI9q3VOCE6jBfJJ+jC6sMUSyVfcKghfp7I8eJ1rMkspNFTr3qFeJ/Xj8z8W2YM5FmhgqyPBRnHBmJihrQkClKDJ9ZgoliNisiY6wwMbasmi3BX/3yOulcNXyv4T9e15t3ZR1VOINzuAQfbqAJD9CCNhCYwjO8wpuTOy/Ou/OxHK045c4p/IHz+QM6jZQIAAAB+XicbVDLSgMxFL1TX7W+Rl26CRbBVZkRQZdFNy4r2Ad0hpJJM21oJhmSTKEM/RM3LhRx65+482/MtLPQ1gOBwzn3ck9OlHKmjed9O5WNza3tnepubW//4PDIPT7paJkpQttEcql6EdaUM0HbhhlOe6miOIk47UaT+8LvTqnSTIonM0tpmOCRYDEj2Fhp4LpBgs04ivNAJkxkej5w617DWwCtE78kdSjRGrhfwVCSLKHCEI617vteasIcK8MIp/NakGmaYjLBI9q3VOCE6jBfJJ+jC6sMUSyVfcKghfp7I8eJ1rMkspNFTr3qFeJ/Xj8z8W2YM5FmhgqyPBRnHBmJihrQkClKDJ9ZgoliNisiY6wwMbasmi3BX/3yOulcNXyv4T9e15t3ZR1VOINzuAQfbqAJD9CCNhCYwjO8wpuTOy/Ou/OxHK045c4p/IHz+QM6jZQI AAAB+XicbVDLSgMxFL1TX7W+Rl26CRbBVZkRQZdFNy4r2Ad0hpJJM21oJhmSTKEM/RM3LhRx65+482/MtLPQ1gOBwzn3ck9OlHKmjed9O5WNza3tnepubW//4PDIPT7paJkpQttEcql6EdaUM0HbhhlOe6miOIk47UaT+8LvTqnSTIonM0tpmOCRYDEj2Fhp4LpBgs04ivNAJkxkej5w617DWwCtE78kdSjRGrhfwVCSLKHCEI617vteasIcK8MIp/NakGmaYjLBI9q3VOCE6jBfJJ+jC6sMUSyVfcKghfp7I8eJ1rMkspNFTr3qFeJ/Xj8z8W2YM5FmhgqyPBRnHBmJihrQkClKDJ9ZgoliNisiY6wwMbasmi3BX/3yOulcNXyv4T9e15t3ZR1VOINzuAQfbqAJD9CCNhCYwjO8wpuTOy/Ou/OxHK045c4p/IHz+QM6jZQIAAAB+XicbVDLSgMxFL1TX7W+Rl26CRbBVZkRQZdFNy4r2Ad0hpJJM21oJhmSTKEM/RM3LhRx65+482/MtLPQ1gOBwzn3ck9OlHKmjed9O5WNza3tnepubW//4PDIPT7paJkpQttEcql6EdaUM0HbhhlOe6miOIk47UaT+8LvTqnSTIonM0tpmOCRYDEj2Fhp4LpBgs04ivNAJkxkej5w617DWwCtE78kdSjRGrhfwVCSLKHCEI617vteasIcK8MIp/NakGmaYjLBI9q3VOCE6jBfJJ+jC6sMUSyVfcKghfp7I8eJ1rMkspNFTr3qFeJ/Xj8z8W2YM5FmhgqyPBRnHBmJihrQkClKDJ9ZgoliNisiY6wwMbasmi3BX/3yOulcNXyv4T9e15t3ZR1VOINzuAQfbqAJD9CCNhCYwjO8wpuTOy/Ou/OxHK045c4p/IHz+QM6jZQIAAAB+XicbVDLSgMxFL1TX7W+Rl26CRbBVZkRQZdFNy4r2Ad0hpJJM21oJhmSTKEM/RM3LhRx65+482/MtLPQ1gOBwzn3ck9OlHKmjed9O5WNza3tnepubW//4PDIPT7paJkpQttEcql6EdaUM0HbhhlOe6miOIk47UaT+8LvTqnSTIonM0tpmOCRYDEj2Fhp4LpBgs04ivNAJkxkej5w617DWwCtE78kdSjRGrhfwVCSLKHCEI617vteasIcK8MIp/NakGmaYjLBI9q3VOCE6jBfJJ+jC6sMUSyVfcKghfp7I8eJ1rMkspNFTr3qFeJ/Xj8z8W2YM5FmhgqyPBRnHBmJihrQkClKDJ9ZgoliNisiY6wwMbasmi3BX/3yOulcNXyv4T9e15t3ZR1VOINzuAQfbqAJD9CCNhCYwjO8wpuTOy/Ou/OxHK045c4p/IHz+QM6jZQIAAAB+XicbVDLSgMxFL1TX7W+Rl26CRbBVZkRQZdFNy4r2Ad0hpJJM21oJhmSTKEM/RM3LhRx65+482/MtLPQ1gOBwzn3ck9OlHKmjed9O5WNza3tnepubW//4PDIPT7paJkpQttEcql6EdaUM0HbhhlOe6miOIk47UaT+8LvTqnSTIonM0tpmOCRYDEj2Fhp4LpBgs04ivNAJkxkej5w617DWwCtE78kdSjRGrhfwVCSLKHCEI617vteasIcK8MIp/NakGmaYjLBI9q3VOCE6jBfJJ+jC6sMUSyVfcKghfp7I8eJ1rMkspNFTr3qFeJ/Xj8z8W2YM5FmhgqyPBRnHBmJihrQkClKDJ9ZgoliNisiY6wwMbasmi3BX/3yOulcNXyv4T9e15t3ZR1VOINzuAQfbqAJD9CCNhCYwjO8wpuTOy/Ou/OxHK045c4p/IHz+QM6jZQI...AAAB7XicbVBNS8NAEJ34WetX1aOXYBE8lUQEPRa9eKxgP6ANZbPZtGs3u2F3Uiih/8GLB0W8+n+8+W/ctjlo64OBx3szzMwLU8ENet63s7a+sbm1Xdop7+7tHxxWjo5bRmWasiZVQulOSAwTXLImchSsk2pGklCwdji6m/ntMdOGK/mIk5QFCRlIHnNK0Eqt3jhSaPqVqlfz5nBXiV+QKhRo9CtfvUjRLGESqSDGdH0vxSAnGjkVbFruZYalhI7IgHUtlSRhJsjn107dc6tEbqy0LYnuXP09kZPEmEkS2s6E4NAsezPxP6+bYXwT5FymGTJJF4viTLio3NnrbsQ1oygmlhCqub3VpUOiCUUbUNmG4C+/vEpalzXfq/kPV9X6bRFHCU7hDC7Ah2uowz00oAkUnuAZXuHNUc6L8+58LFrXnGLmBP7A+fwBy2+PQg==AAAB7XicbVBNS8NAEJ34WetX1aOXYBE8lUQEPRa9eKxgP6ANZbPZtGs3u2F3Uiih/8GLB0W8+n+8+W/ctjlo64OBx3szzMwLU8ENet63s7a+sbm1Xdop7+7tHxxWjo5bRmWasiZVQulOSAwTXLImchSsk2pGklCwdji6m/ntMdOGK/mIk5QFCRlIHnNK0Eqt3jhSaPqVqlfz5nBXiV+QKhRo9CtfvUjRLGESqSDGdH0vxSAnGjkVbFruZYalhI7IgHUtlSRhJsjn107dc6tEbqy0LYnuXP09kZPEmEkS2s6E4NAsezPxP6+bYXwT5FymGTJJF4viTLio3NnrbsQ1oygmlhCqub3VpUOiCUUbUNmG4C+/vEpalzXfq/kPV9X6bRFHCU7hDC7Ah2uowz00oAkUnuAZXuHNUc6L8+58LFrXnGLmBP7A+fwBy2+PQg==AAAB7XicbVBNS8NAEJ34WetX1aOXYBE8lUQEPRa9eKxgP6ANZbPZtGs3u2F3Uiih/8GLB0W8+n+8+W/ctjlo64OBx3szzMwLU8ENet63s7a+sbm1Xdop7+7tHxxWjo5bRmWasiZVQulOSAwTXLImchSsk2pGklCwdji6m/ntMdOGK/mIk5QFCRlIHnNK0Eqt3jhSaPqVqlfz5nBXiV+QKhRo9CtfvUjRLGESqSDGdH0vxSAnGjkVbFruZYalhI7IgHUtlSRhJsjn107dc6tEbqy0LYnuXP09kZPEmEkS2s6E4NAsezPxP6+bYXwT5FymGTJJF4viTLio3NnrbsQ1oygmlhCqub3VpUOiCUUbUNmG4C+/vEpalzXfq/kPV9X6bRFHCU7hDC7Ah2uowz00oAkUnuAZXuHNUc6L8+58LFrXnGLmBP7A+fwBy2+PQg==AAAB7XicbVBNS8NAEJ34WetX1aOXYBE8lUQEPRa9eKxgP6ANZbPZtGs3u2F3Uiih/8GLB0W8+n+8+W/ctjlo64OBx3szzMwLU8ENet63s7a+sbm1Xdop7+7tHxxWjo5bRmWasiZVQulOSAwTXLImchSsk2pGklCwdji6m/ntMdOGK/mIk5QFCRlIHnNK0Eqt3jhSaPqVqlfz5nBXiV+QKhRo9CtfvUjRLGESqSDGdH0vxSAnGjkVbFruZYalhI7IgHUtlSRhJsjn107dc6tEbqy0LYnuXP09kZPEmEkS2s6E4NAsezPxP6+bYXwT5FymGTJJF4viTLio3NnrbsQ1oygmlhCqub3VpUOiCUUbUNmG4C+/vEpalzXfq/kPV9X6bRFHCU7hDC7Ah2uowz00oAkUnuAZXuHNUc6L8+58LFrXnGLmBP7A+fwBy2+PQg==ContrastModuleAAAB+HicbVBNS8NAEN34WetHox69LBbBU0l60WOxFy9CBfsBbSibzaZdutkNuxOhhv4SLx4U8epP8ea/cdvmoK0PBh7vzTAzL0wFN+B5387G5tb2zm5pr7x/cHhUcY9POkZlmrI2VULpXkgME1yyNnAQrJdqRpJQsG44ac797iPThiv5ANOUBQkZSR5zSsBKQ7fSVBI0MYDvVJQJNnSrXs1bAK8TvyBVVKA1dL8GkaJZwiRQQYzp+14KQU40cCrYrDzIDEsJnZAR61sqScJMkC8On+ELq0Q4VtqWBLxQf0/kJDFmmoS2MyEwNqveXPzP62cQXwc5l2kGTNLlojgTGBSep4AjrhkFMbWEUM3trZiOiSYUbFZlG4K/+vI66dRrvlfz7+vVxk0RRwmdoXN0iXx0hRroFrVQG1GUoWf0it6cJ+fFeXc+lq0bTjFziv7A+fwBqyKTEw==AAAB+HicbVBNS8NAEN34WetHox69LBbBU0l60WOxFy9CBfsBbSibzaZdutkNuxOhhv4SLx4U8epP8ea/cdvmoK0PBh7vzTAzL0wFN+B5387G5tb2zm5pr7x/cHhUcY9POkZlmrI2VULpXkgME1yyNnAQrJdqRpJQsG44ac797iPThiv5ANOUBQkZSR5zSsBKQ7fSVBI0MYDvVJQJNnSrXs1bAK8TvyBVVKA1dL8GkaJZwiRQQYzp+14KQU40cCrYrDzIDEsJnZAR61sqScJMkC8On+ELq0Q4VtqWBLxQf0/kJDFmmoS2MyEwNqveXPzP62cQXwc5l2kGTNLlojgTGBSep4AjrhkFMbWEUM3trZiOiSYUbFZlG4K/+vI66dRrvlfz7+vVxk0RRwmdoXN0iXx0hRroFrVQG1GUoWf0it6cJ+fFeXc+lq0bTjFziv7A+fwBqyKTEw==AAAB+HicbVBNS8NAEN34WetHox69LBbBU0l60WOxFy9CBfsBbSibzaZdutkNuxOhhv4SLx4U8epP8ea/cdvmoK0PBh7vzTAzL0wFN+B5387G5tb2zm5pr7x/cHhUcY9POkZlmrI2VULpXkgME1yyNnAQrJdqRpJQsG44ac797iPThiv5ANOUBQkZSR5zSsBKQ7fSVBI0MYDvVJQJNnSrXs1bAK8TvyBVVKA1dL8GkaJZwiRQQYzp+14KQU40cCrYrDzIDEsJnZAR61sqScJMkC8On+ELq0Q4VtqWBLxQf0/kJDFmmoS2MyEwNqveXPzP62cQXwc5l2kGTNLlojgTGBSep4AjrhkFMbWEUM3trZiOiSYUbFZlG4K/+vI66dRrvlfz7+vVxk0RRwmdoXN0iXx0hRroFrVQG1GUoWf0it6cJ+fFeXc+lq0bTjFziv7A+fwBqyKTEw==AAAB+HicbVBNS8NAEN34WetHox69LBbBU0l60WOxFy9CBfsBbSibzaZdutkNuxOhhv4SLx4U8epP8ea/cdvmoK0PBh7vzTAzL0wFN+B5387G5tb2zm5pr7x/cHhUcY9POkZlmrI2VULpXkgME1yyNnAQrJdqRpJQsG44ac797iPThiv5ANOUBQkZSR5zSsBKQ7fSVBI0MYDvVJQJNnSrXs1bAK8TvyBVVKA1dL8GkaJZwiRQQYzp+14KQU40cCrYrDzIDEsJnZAR61sqScJMkC8On+ELq0Q4VtqWBLxQf0/kJDFmmoS2MyEwNqveXPzP62cQXwc5l2kGTNLlojgTGBSep4AjrhkFMbWEUM3trZiOiSYUbFZlG4K/+vI66dRrvlfz7+vVxk0RRwmdoXN0iXx0hRroFrVQG1GUoWf0it6cJ+fFeXc+lq0bTjFziv7A+fwBqyKTEw==(c)AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoMQm3CXRsugjWVE8wHJEfY2e8mSvb1jd04IR36CjYUitv4iO/+Nm+QKTXww8Hhvhpl5QSKFQdf9dgobm1vbO8Xd0t7+weFR+fikbeJUM95isYx1N6CGS6F4CwVK3k00p1EgeSeY3M79zhPXRsTqEacJ9yM6UiIUjKKVHqrsclCuuDV3AbJOvJxUIEdzUP7qD2OWRlwhk9SYnucm6GdUo2CSz0r91PCEsgkd8Z6likbc+Nni1Bm5sMqQhLG2pZAs1N8TGY2MmUaB7Ywojs2qNxf/83ophtd+JlSSIldsuShMJcGYzP8mQ6E5Qzm1hDIt7K2EjammDG06JRuCt/ryOmnXa55b8+7rlcZNHkcRzuAcquDBFTTgDprQAgYjeIZXeHOk8+K8Ox/L1oKTz5zCHzifP4sVjUo=AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoMQm3CXRsugjWVE8wHJEfY2e8mSvb1jd04IR36CjYUitv4iO/+Nm+QKTXww8Hhvhpl5QSKFQdf9dgobm1vbO8Xd0t7+weFR+fikbeJUM95isYx1N6CGS6F4CwVK3k00p1EgeSeY3M79zhPXRsTqEacJ9yM6UiIUjKKVHqrsclCuuDV3AbJOvJxUIEdzUP7qD2OWRlwhk9SYnucm6GdUo2CSz0r91PCEsgkd8Z6likbc+Nni1Bm5sMqQhLG2pZAs1N8TGY2MmUaB7Ywojs2qNxf/83ophtd+JlSSIldsuShMJcGYzP8mQ6E5Qzm1hDIt7K2EjammDG06JRuCt/ryOmnXa55b8+7rlcZNHkcRzuAcquDBFTTgDprQAgYjeIZXeHOk8+K8Ox/L1oKTz5zCHzifP4sVjUo=AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoMQm3CXRsugjWVE8wHJEfY2e8mSvb1jd04IR36CjYUitv4iO/+Nm+QKTXww8Hhvhpl5QSKFQdf9dgobm1vbO8Xd0t7+weFR+fikbeJUM95isYx1N6CGS6F4CwVK3k00p1EgeSeY3M79zhPXRsTqEacJ9yM6UiIUjKKVHqrsclCuuDV3AbJOvJxUIEdzUP7qD2OWRlwhk9SYnucm6GdUo2CSz0r91PCEsgkd8Z6likbc+Nni1Bm5sMqQhLG2pZAs1N8TGY2MmUaB7Ywojs2qNxf/83ophtd+JlSSIldsuShMJcGYzP8mQ6E5Qzm1hDIt7K2EjammDG06JRuCt/ryOmnXa55b8+7rlcZNHkcRzuAcquDBFTTgDprQAgYjeIZXeHOk8+K8Ox/L1oKTz5zCHzifP4sVjUo=AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoMQm3CXRsugjWVE8wHJEfY2e8mSvb1jd04IR36CjYUitv4iO/+Nm+QKTXww8Hhvhpl5QSKFQdf9dgobm1vbO8Xd0t7+weFR+fikbeJUM95isYx1N6CGS6F4CwVK3k00p1EgeSeY3M79zhPXRsTqEacJ9yM6UiIUjKKVHqrsclCuuDV3AbJOvJxUIEdzUP7qD2OWRlwhk9SYnucm6GdUo2CSz0r91PCEsgkd8Z6likbc+Nni1Bm5sMqQhLG2pZAs1N8TGY2MmUaB7Ywojs2qNxf/83ophtd+JlSSIldsuShMJcGYzP8mQ6E5Qzm1hDIt7K2EjammDG06JRuCt/ryOmnXa55b8+7rlcZNHkcRzuAcquDBFTTgDprQAgYjeIZXeHOk8+K8Ox/L1oKTz5zCHzifP4sVjUo=(b)AAAB6nicbVA9SwNBEJ2LXzF+RS1tFoMQm3CXRsugjWVE8wHJEfY2e8mSvb1jd04IR36CjYUitv4iO/+Nm+QKTXww8Hhvhpl5QSKFQdf9dgobm1vbO8Xd0t7+weFR+fikbeJUM95isYx1N6CGS6F4CwVK3k00p1EgeSeY3M79zhPXRsTqEacJ9yM6UiIUjKKVHqrB5aBccWvuAmSdeDmpQI7moPzVH8YsjbhCJqkxPc9N0M+oRsEkn5X6qeEJZRM64j1LFY248bPFqTNyYZUhCWNtSyFZqL8nMhoZM40C2xlRHJtVby7+5/VSDK/9TKgkRa7YclGYSoIxmf9NhkJzhnJqCWVa2FsJG1NNGdp0SjYEb/XlddKu1zy35t3XK42bPI4inME5VMGDK2jAHTShBQxG8Ayv8OZI58V5dz6WrQUnnzmFP3A+fwCJkI1JAAAB6nicbVA9SwNBEJ2LXzF+RS1tFoMQm3CXRsugjWVE8wHJEfY2e8mSvb1jd04IR36CjYUitv4iO/+Nm+QKTXww8Hhvhpl5QSKFQdf9dgobm1vbO8Xd0t7+weFR+fikbeJUM95isYx1N6CGS6F4CwVK3k00p1EgeSeY3M79zhPXRsTqEacJ9yM6UiIUjKKVHqrB5aBccWvuAmSdeDmpQI7moPzVH8YsjbhCJqkxPc9N0M+oRsEkn5X6qeEJZRM64j1LFY248bPFqTNyYZUhCWNtSyFZqL8nMhoZM40C2xlRHJtVby7+5/VSDK/9TKgkRa7YclGYSoIxmf9NhkJzhnJqCWVa2FsJG1NNGdp0SjYEb/XlddKu1zy35t3XK42bPI4inME5VMGDK2jAHTShBQxG8Ayv8OZI58V5dz6WrQUnnzmFP3A+fwCJkI1JAAAB6nicbVA9SwNBEJ2LXzF+RS1tFoMQm3CXRsugjWVE8wHJEfY2e8mSvb1jd04IR36CjYUitv4iO/+Nm+QKTXww8Hhvhpl5QSKFQdf9dgobm1vbO8Xd0t7+weFR+fikbeJUM95isYx1N6CGS6F4CwVK3k00p1EgeSeY3M79zhPXRsTqEacJ9yM6UiIUjKKVHqrB5aBccWvuAmSdeDmpQI7moPzVH8YsjbhCJqkxPc9N0M+oRsEkn5X6qeEJZRM64j1LFY248bPFqTNyYZUhCWNtSyFZqL8nMhoZM40C2xlRHJtVby7+5/VSDK/9TKgkRa7YclGYSoIxmf9NhkJzhnJqCWVa2FsJG1NNGdp0SjYEb/XlddKu1zy35t3XK42bPI4inME5VMGDK2jAHTShBQxG8Ayv8OZI58V5dz6WrQUnnzmFP3A+fwCJkI1JAAAB6nicbVA9SwNBEJ2LXzF+RS1tFoMQm3CXRsugjWVE8wHJEfY2e8mSvb1jd04IR36CjYUitv4iO/+Nm+QKTXww8Hhvhpl5QSKFQdf9dgobm1vbO8Xd0t7+weFR+fikbeJUM95isYx1N6CGS6F4CwVK3k00p1EgeSeY3M79zhPXRsTqEacJ9yM6UiIUjKKVHqrB5aBccWvuAmSdeDmpQI7moPzVH8YsjbhCJqkxPc9N0M+oRsEkn5X6qeEJZRM64j1LFY248bPFqTNyYZUhCWNtSyFZqL8nMhoZM40C2xlRHJtVby7+5/VSDK/9TKgkRa7YclGYSoIxmf9NhkJzhnJqCWVa2FsJG1NNGdp0SjYEb/XlddKu1zy35t3XK42bPI4inME5VMGDK2jAHTShBQxG8Ayv8OZI58V5dz6WrQUnnzmFP3A+fwCJkI1J(Gumbel-)SoftMaxAAAB+XicbVA9SwNBEN3zM8avU0ubxSDEwnCXRsughTZCRPMBSQh7m7lkyd7usbsXDEf+iY2FIrb+Ezv/jZvkCk18MPB4b4aZeUHMmTae9+2srK6tb2zmtvLbO7t7++7BYV3LRFGoUcmlagZEA2cCaoYZDs1YAYkCDo1geD31GyNQmknxaMYxdCLSFyxklBgrdV23eJNEAfDzswcZmjvy1HULXsmbAS8TPyMFlKHadb/aPUmTCIShnGjd8r3YdFKiDKMcJvl2oiEmdEj60LJUkAh0J51dPsGnVunhUCpbwuCZ+nsiJZHW4yiwnRExA73oTcX/vFZiwstOykScGBB0vihMODYST2PAPaaAGj62hFDF7K2YDogi1Niw8jYEf/HlZVIvl3yv5N+XC5WrLI4cOkYnqIh8dIEq6BZVUQ1RNELP6BW9Oanz4rw7H/PWFSebOUJ/4Hz+AHqekuU=AAAB+XicbVA9SwNBEN3zM8avU0ubxSDEwnCXRsughTZCRPMBSQh7m7lkyd7usbsXDEf+iY2FIrb+Ezv/jZvkCk18MPB4b4aZeUHMmTae9+2srK6tb2zmtvLbO7t7++7BYV3LRFGoUcmlagZEA2cCaoYZDs1YAYkCDo1geD31GyNQmknxaMYxdCLSFyxklBgrdV23eJNEAfDzswcZmjvy1HULXsmbAS8TPyMFlKHadb/aPUmTCIShnGjd8r3YdFKiDKMcJvl2oiEmdEj60LJUkAh0J51dPsGnVunhUCpbwuCZ+nsiJZHW4yiwnRExA73oTcX/vFZiwstOykScGBB0vihMODYST2PAPaaAGj62hFDF7K2YDogi1Niw8jYEf/HlZVIvl3yv5N+XC5WrLI4cOkYnqIh8dIEq6BZVUQ1RNELP6BW9Oanz4rw7H/PWFSebOUJ/4Hz+AHqekuU=AAAB+XicbVA9SwNBEN3zM8avU0ubxSDEwnCXRsughTZCRPMBSQh7m7lkyd7usbsXDEf+iY2FIrb+Ezv/jZvkCk18MPB4b4aZeUHMmTae9+2srK6tb2zmtvLbO7t7++7BYV3LRFGoUcmlagZEA2cCaoYZDs1YAYkCDo1geD31GyNQmknxaMYxdCLSFyxklBgrdV23eJNEAfDzswcZmjvy1HULXsmbAS8TPyMFlKHadb/aPUmTCIShnGjd8r3YdFKiDKMcJvl2oiEmdEj60LJUkAh0J51dPsGnVunhUCpbwuCZ+nsiJZHW4yiwnRExA73oTcX/vFZiwstOykScGBB0vihMODYST2PAPaaAGj62hFDF7K2YDogi1Niw8jYEf/HlZVIvl3yv5N+XC5WrLI4cOkYnqIh8dIEq6BZVUQ1RNELP6BW9Oanz4rw7H/PWFSebOUJ/4Hz+AHqekuU=AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4AAAB7nicbZBLSwMxFIXv+Ky16ujWTbAIdWGZcaNLwYVuhIr2Ae1QMmmmDc1jSDLFMvSfuHGhiD/Hnf/G9LHQ1gOBj3MS7s2JU86MDYJvb219Y3Nru7BT3C3t7R/4h6WGUZkmtE4UV7oVY0M5k7RumeW0lWqKRcxpMx7eTPPmiGrDlHyy45RGAvclSxjB1lld36/cZiKm/PzsUSX2Hj93/XJQDWZCqxAuoAwL1br+V6enSCaotIRjY9phkNoox9oywumk2MkMTTEZ4j5tO5RYUBPls80n6NQ5PZQo7Y60aOb+fpFjYcxYxO6mwHZglrOp+V/WzmxyFeVMppmlkswHJRlHVqFpDajHNCWWjx1gopnbFZEB1phYV1bRlRAuf3kVGhfVMKiGDwEU4BhOoAIhXMI13EEN6kBgBC/wBu9e7r16H/O61rxFb0fwR97nDyiNkYM=AAAB7nicbZBLSwMxFIXv+Ky16ujWTbAIdWGZcaNLwYVuhIr2Ae1QMmmmDc1jSDLFMvSfuHGhiD/Hnf/G9LHQ1gOBj3MS7s2JU86MDYJvb219Y3Nru7BT3C3t7R/4h6WGUZkmtE4UV7oVY0M5k7RumeW0lWqKRcxpMx7eTPPmiGrDlHyy45RGAvclSxjB1lld36/cZiKm/PzsUSX2Hj93/XJQDWZCqxAuoAwL1br+V6enSCaotIRjY9phkNoox9oywumk2MkMTTEZ4j5tO5RYUBPls80n6NQ5PZQo7Y60aOb+fpFjYcxYxO6mwHZglrOp+V/WzmxyFeVMppmlkswHJRlHVqFpDajHNCWWjx1gopnbFZEB1phYV1bRlRAuf3kVGhfVMKiGDwEU4BhOoAIhXMI13EEN6kBgBC/wBu9e7r16H/O61rxFb0fwR97nDyiNkYM=AAAB+XicbVDLSgNBEJyNrxhfqx69LAYhHgy7XvQY9KAXIaJ5QBLC7KQ3GTKPZWY2GJb8iRcPinj1T7z5N06SPWhiQUNR1U13Vxgzqo3vfzu5ldW19Y38ZmFre2d3z90/qGuZKAI1IplUzRBrYFRAzVDDoBkrwDxk0AiH11O/MQKlqRSPZhxDh+O+oBEl2Fip67qlm4SHwM5OH2Rk7vBT1y36ZX8Gb5kEGSmiDNWu+9XuSZJwEIYwrHUr8GPTSbEylDCYFNqJhhiTIe5Dy1KBOehOOrt84p1YpedFUtkSxpupvydSzLUe89B2cmwGetGbiv95rcREl52UijgxIMh8UZQwz0hvGoPXowqIYWNLMFHU3uqRAVaYGBtWwYYQLL68TOrn5cAvB/d+sXKVxZFHR+gYlVCALlAF3aIqqiGCRugZvaI3J3VenHfnY96ac7KZQ/QHzucPef6S4w==AAAB+XicbVA9SwNBEN3zM8avU0ubxSDEwnCXRsughTZCRPMBSQh7m7lkyd7usbsXDEf+iY2FIrb+Ezv/jZvkCk18MPB4b4aZeUHMmTae9+2srK6tb2zmtvLbO7t7++7BYV3LRFGoUcmlagZEA2cCaoYZDs1YAYkCDo1geD31GyNQmknxaMYxdCLSFyxklBgrdV23eJNEAfDzswcZmjvy1HULXsmbAS8TPyMFlKHadb/aPUmTCIShnGjd8r3YdFKiDKMcJvl2oiEmdEj60LJUkAh0J51dPsGnVunhUCpbwuCZ+nsiJZHW4yiwnRExA73oTcX/vFZiwstOykScGBB0vihMODYST2PAPaaAGj62hFDF7K2YDogi1Niw8jYEf/HlZVIvl3yv5N+XC5WrLI4cOkYnqIh8dIEq6BZVUQ1RNELP6BW9Oanz4rw7H/PWFSebOUJ/4Hz+AHqekuU=AAAB+XicbVA9SwNBEN3zM8avU0ubxSDEwnCXRsughTZCRPMBSQh7m7lkyd7usbsXDEf+iY2FIrb+Ezv/jZvkCk18MPB4b4aZeUHMmTae9+2srK6tb2zmtvLbO7t7++7BYV3LRFGoUcmlagZEA2cCaoYZDs1YAYkCDo1geD31GyNQmknxaMYxdCLSFyxklBgrdV23eJNEAfDzswcZmjvy1HULXsmbAS8TPyMFlKHadb/aPUmTCIShnGjd8r3YdFKiDKMcJvl2oiEmdEj60LJUkAh0J51dPsGnVunhUCpbwuCZ+nsiJZHW4yiwnRExA73oTcX/vFZiwstOykScGBB0vihMODYST2PAPaaAGj62hFDF7K2YDogi1Niw8jYEf/HlZVIvl3yv5N+XC5WrLI4cOkYnqIh8dIEq6BZVUQ1RNELP6BW9Oanz4rw7H/PWFSebOUJ/4Hz+AHqekuU=AAAB+XicbVA9SwNBEN3zM8avU0ubxSDEwnCXRsughTZCRPMBSQh7m7lkyd7usbsXDEf+iY2FIrb+Ezv/jZvkCk18MPB4b4aZeUHMmTae9+2srK6tb2zmtvLbO7t7++7BYV3LRFGoUcmlagZEA2cCaoYZDs1YAYkCDo1geD31GyNQmknxaMYxdCLSFyxklBgrdV23eJNEAfDzswcZmjvy1HULXsmbAS8TPyMFlKHadb/aPUmTCIShnGjd8r3YdFKiDKMcJvl2oiEmdEj60LJUkAh0J51dPsGnVunhUCpbwuCZ+nsiJZHW4yiwnRExA73oTcX/vFZiwstOykScGBB0vihMODYST2PAPaaAGj62hFDF7K2YDogi1Niw8jYEf/HlZVIvl3yv5N+XC5WrLI4cOkYnqIh8dIEq6BZVUQ1RNELP6BW9Oanz4rw7H/PWFSebOUJ/4Hz+AHqekuU=AAAB+XicbVA9SwNBEN3zM8avU0ubxSDEwnCXRsughTZCRPMBSQh7m7lkyd7usbsXDEf+iY2FIrb+Ezv/jZvkCk18MPB4b4aZeUHMmTae9+2srK6tb2zmtvLbO7t7++7BYV3LRFGoUcmlagZEA2cCaoYZDs1YAYkCDo1geD31GyNQmknxaMYxdCLSFyxklBgrdV23eJNEAfDzswcZmjvy1HULXsmbAS8TPyMFlKHadb/aPUmTCIShnGjd8r3YdFKiDKMcJvl2oiEmdEj60LJUkAh0J51dPsGnVunhUCpbwuCZ+nsiJZHW4yiwnRExA73oTcX/vFZiwstOykScGBB0vihMODYST2PAPaaAGj62hFDF7K2YDogi1Niw8jYEf/HlZVIvl3yv5N+XC5WrLI4cOkYnqIh8dIEq6BZVUQ1RNELP6BW9Oanz4rw7H/PWFSebOUJ/4Hz+AHqekuU=AAAB+XicbVA9SwNBEN3zM8avU0ubxSDEwnCXRsughTZCRPMBSQh7m7lkyd7usbsXDEf+iY2FIrb+Ezv/jZvkCk18MPB4b4aZeUHMmTae9+2srK6tb2zmtvLbO7t7++7BYV3LRFGoUcmlagZEA2cCaoYZDs1YAYkCDo1geD31GyNQmknxaMYxdCLSFyxklBgrdV23eJNEAfDzswcZmjvy1HULXsmbAS8TPyMFlKHadb/aPUmTCIShnGjd8r3YdFKiDKMcJvl2oiEmdEj60LJUkAh0J51dPsGnVunhUCpbwuCZ+nsiJZHW4yiwnRExA73oTcX/vFZiwstOykScGBB0vihMODYST2PAPaaAGj62hFDF7K2YDogi1Niw8jYEf/HlZVIvl3yv5N+XC5WrLI4cOkYnqIh8dIEq6BZVUQ1RNELP6BW9Oanz4rw7H/PWFSebOUJ/4Hz+AHqekuU=AAAB+XicbVA9SwNBEN3zM8avU0ubxSDEwnCXRsughTZCRPMBSQh7m7lkyd7usbsXDEf+iY2FIrb+Ezv/jZvkCk18MPB4b4aZeUHMmTae9+2srK6tb2zmtvLbO7t7++7BYV3LRFGoUcmlagZEA2cCaoYZDs1YAYkCDo1geD31GyNQmknxaMYxdCLSFyxklBgrdV23eJNEAfDzswcZmjvy1HULXsmbAS8TPyMFlKHadb/aPUmTCIShnGjd8r3YdFKiDKMcJvl2oiEmdEj60LJUkAh0J51dPsGnVunhUCpbwuCZ+nsiJZHW4yiwnRExA73oTcX/vFZiwstOykScGBB0vihMODYST2PAPaaAGj62hFDF7K2YDogi1Niw8jYEf/HlZVIvl3yv5N+XC5WrLI4cOkYnqIh8dIEq6BZVUQ1RNELP6BW9Oanz4rw7H/PWFSebOUJ/4Hz+AHqekuU=OAAAB8nicbVDLSsNAFL2pr1pfVZdugkVwVRIRdFl0484K9gFtKJPppB06mQkzN0IJ/Qw3LhRx69e482+ctFlo64GBwzn3MueeMBHcoOd9O6W19Y3NrfJ2ZWd3b/+genjUNirVlLWoEkp3Q2KY4JK1kKNg3UQzEoeCdcLJbe53npg2XMlHnCYsiMlI8ohTglbq9WOCY0pEdj8bVGte3ZvDXSV+QWpQoDmofvWHiqYxk0gFMabnewkGGdHIqWCzSj81LCF0QkasZ6kkMTNBNo88c8+sMnQjpe2T6M7V3xsZiY2ZxqGdzCOaZS8X//N6KUbXQcZlkiKTdPFRlAoXlZvf7w65ZhTF1BJCNbdZXTommlC0LVVsCf7yyaukfVH3vbr/cFlr3BR1lOEETuEcfLiCBtxBE1pAQcEzvMKbg86L8+58LEZLTrFzDH/gfP4AhZSRZQ==AAAB8nicbVDLSsNAFL2pr1pfVZdugkVwVRIRdFl0484K9gFtKJPppB06mQkzN0IJ/Qw3LhRx69e482+ctFlo64GBwzn3MueeMBHcoOd9O6W19Y3NrfJ2ZWd3b/+genjUNirVlLWoEkp3Q2KY4JK1kKNg3UQzEoeCdcLJbe53npg2XMlHnCYsiMlI8ohTglbq9WOCY0pEdj8bVGte3ZvDXSV+QWpQoDmofvWHiqYxk0gFMabnewkGGdHIqWCzSj81LCF0QkasZ6kkMTNBNo88c8+sMnQjpe2T6M7V3xsZiY2ZxqGdzCOaZS8X//N6KUbXQcZlkiKTdPFRlAoXlZvf7w65ZhTF1BJCNbdZXTommlC0LVVsCf7yyaukfVH3vbr/cFlr3BR1lOEETuEcfLiCBtxBE1pAQcEzvMKbg86L8+58LEZLTrFzDH/gfP4AhZSRZQ==AAAB8nicbVDLSsNAFL2pr1pfVZdugkVwVRIRdFl0484K9gFtKJPppB06mQkzN0IJ/Qw3LhRx69e482+ctFlo64GBwzn3MueeMBHcoOd9O6W19Y3NrfJ2ZWd3b/+genjUNirVlLWoEkp3Q2KY4JK1kKNg3UQzEoeCdcLJbe53npg2XMlHnCYsiMlI8ohTglbq9WOCY0pEdj8bVGte3ZvDXSV+QWpQoDmofvWHiqYxk0gFMabnewkGGdHIqWCzSj81LCF0QkasZ6kkMTNBNo88c8+sMnQjpe2T6M7V3xsZiY2ZxqGdzCOaZS8X//N6KUbXQcZlkiKTdPFRlAoXlZvf7w65ZhTF1BJCNbdZXTommlC0LVVsCf7yyaukfVH3vbr/cFlr3BR1lOEETuEcfLiCBtxBE1pAQcEzvMKbg86L8+58LEZLTrFzDH/gfP4AhZSRZQ==AAAB8nicbVDLSsNAFL2pr1pfVZdugkVwVRIRdFl0484K9gFtKJPppB06mQkzN0IJ/Qw3LhRx69e482+ctFlo64GBwzn3MueeMBHcoOd9O6W19Y3NrfJ2ZWd3b/+genjUNirVlLWoEkp3Q2KY4JK1kKNg3UQzEoeCdcLJbe53npg2XMlHnCYsiMlI8ohTglbq9WOCY0pEdj8bVGte3ZvDXSV+QWpQoDmofvWHiqYxk0gFMabnewkGGdHIqWCzSj81LCF0QkasZ6kkMTNBNo88c8+sMnQjpe2T6M7V3xsZiY2ZxqGdzCOaZS8X//N6KUbXQcZlkiKTdPFRlAoXlZvf7w65ZhTF1BJCNbdZXTommlC0LVVsCf7yyaukfVH3vbr/cFlr3BR1lOEETuEcfLiCBtxBE1pAQcEzvMKbg86L8+58LEZLTrFzDH/gfP4AhZSRZQ==O[a1AAAB/XicbVDLSsNAFJ3UV62v+Ni5GSyCq5KIoMuiG3dWsLXQhHAznbRDJ5MwMxFqKP6KGxeKuPU/3Pk3TtostPXAwOGce7lnTphyprTjfFuVpeWV1bXqem1jc2t7x97d66gkk4S2ScIT2Q1BUc4EbWumOe2mkkIccnofjq4K//6BSsUScafHKfVjGAgWMQLaSIF94MWghwR4fjPBHslSDIEb2HWn4UyBF4lbkjoq0QrsL6+fkCymQhMOSvVcJ9V+DlIzwumk5mWKpkBGMKA9QwXEVPn5NP0EHxulj6NEmic0nqq/N3KIlRrHoZkssqp5rxD/83qZji78nIk001SQ2aEo41gnuKgC95mkRPOxIUAkM1kxGYIEok1hNVOCO//lRdI5bbhOw709qzcvyzqq6BAdoRPkonPURNeohdqIoEf0jF7Rm/VkvVjv1sdstGKVO/voD6zPH8nXlMU=AAAB/XicbVDLSsNAFJ3UV62v+Ni5GSyCq5KIoMuiG3dWsLXQhHAznbRDJ5MwMxFqKP6KGxeKuPU/3Pk3TtostPXAwOGce7lnTphyprTjfFuVpeWV1bXqem1jc2t7x97d66gkk4S2ScIT2Q1BUc4EbWumOe2mkkIccnofjq4K//6BSsUScafHKfVjGAgWMQLaSIF94MWghwR4fjPBHslSDIEb2HWn4UyBF4lbkjoq0QrsL6+fkCymQhMOSvVcJ9V+DlIzwumk5mWKpkBGMKA9QwXEVPn5NP0EHxulj6NEmic0nqq/N3KIlRrHoZkssqp5rxD/83qZji78nIk001SQ2aEo41gnuKgC95mkRPOxIUAkM1kxGYIEok1hNVOCO//lRdI5bbhOw709qzcvyzqq6BAdoRPkonPURNeohdqIoEf0jF7Rm/VkvVjv1sdstGKVO/voD6zPH8nXlMU=AAAB/XicbVDLSsNAFJ3UV62v+Ni5GSyCq5KIoMuiG3dWsLXQhHAznbRDJ5MwMxFqKP6KGxeKuPU/3Pk3TtostPXAwOGce7lnTphyprTjfFuVpeWV1bXqem1jc2t7x97d66gkk4S2ScIT2Q1BUc4EbWumOe2mkkIccnofjq4K//6BSsUScafHKfVjGAgWMQLaSIF94MWghwR4fjPBHslSDIEb2HWn4UyBF4lbkjoq0QrsL6+fkCymQhMOSvVcJ9V+DlIzwumk5mWKpkBGMKA9QwXEVPn5NP0EHxulj6NEmic0nqq/N3KIlRrHoZkssqp5rxD/83qZji78nIk001SQ2aEo41gnuKgC95mkRPOxIUAkM1kxGYIEok1hNVOCO//lRdI5bbhOw709qzcvyzqq6BAdoRPkonPURNeohdqIoEf0jF7Rm/VkvVjv1sdstGKVO/voD6zPH8nXlMU=AAAB/XicbVDLSsNAFJ3UV62v+Ni5GSyCq5KIoMuiG3dWsLXQhHAznbRDJ5MwMxFqKP6KGxeKuPU/3Pk3TtostPXAwOGce7lnTphyprTjfFuVpeWV1bXqem1jc2t7x97d66gkk4S2ScIT2Q1BUc4EbWumOe2mkkIccnofjq4K//6BSsUScafHKfVjGAgWMQLaSIF94MWghwR4fjPBHslSDIEb2HWn4UyBF4lbkjoq0QrsL6+fkCymQhMOSvVcJ9V+DlIzwumk5mWKpkBGMKA9QwXEVPn5NP0EHxulj6NEmic0nqq/N3KIlRrHoZkssqp5rxD/83qZji78nIk001SQ2aEo41gnuKgC95mkRPOxIUAkM1kxGYIEok1hNVOCO//lRdI5bbhOw709qzcvyzqq6BAdoRPkonPURNeohdqIoEf0jF7Rm/VkvVjv1sdstGKVO/voD6zPH8nXlMU=O[a8AAAB/XicbVDLSsNAFJ3UV62v+Ni5GSyCq5KIYJdFN+6sYB/QhHAznbZDJ5MwMxFqKP6KGxeKuPU/3Pk3TtostPXAwOGce7lnTphwprTjfFulldW19Y3yZmVre2d3z94/aKs4lYS2SMxj2Q1BUc4EbWmmOe0mkkIUctoJx9e533mgUrFY3OtJQv0IhoINGAFtpMA+8iLQIwI8u51ij6QJhqAe2FWn5syAl4lbkCoq0AzsL68fkzSiQhMOSvVcJ9F+BlIzwum04qWKJkDGMKQ9QwVEVPnZLP0UnxqljwexNE9oPFN/b2QQKTWJQjOZZ1WLXi7+5/VSPaj7GRNJqqkg80ODlGMd47wK3GeSEs0nhgCRzGTFZAQSiDaFVUwJ7uKXl0n7vOY6Nffuotq4Kuooo2N0gs6Qiy5RA92gJmohgh7RM3pFb9aT9WK9Wx/z0ZJV7ByiP7A+fwDUc5TMAAAB/XicbVDLSsNAFJ3UV62v+Ni5GSyCq5KIYJdFN+6sYB/QhHAznbZDJ5MwMxFqKP6KGxeKuPU/3Pk3TtostPXAwOGce7lnTphwprTjfFulldW19Y3yZmVre2d3z94/aKs4lYS2SMxj2Q1BUc4EbWmmOe0mkkIUctoJx9e533mgUrFY3OtJQv0IhoINGAFtpMA+8iLQIwI8u51ij6QJhqAe2FWn5syAl4lbkCoq0AzsL68fkzSiQhMOSvVcJ9F+BlIzwum04qWKJkDGMKQ9QwVEVPnZLP0UnxqljwexNE9oPFN/b2QQKTWJQjOZZ1WLXi7+5/VSPaj7GRNJqqkg80ODlGMd47wK3GeSEs0nhgCRzGTFZAQSiDaFVUwJ7uKXl0n7vOY6Nffuotq4Kuooo2N0gs6Qiy5RA92gJmohgh7RM3pFb9aT9WK9Wx/z0ZJV7ByiP7A+fwDUc5TMAAAB/XicbVDLSsNAFJ3UV62v+Ni5GSyCq5KIYJdFN+6sYB/QhHAznbZDJ5MwMxFqKP6KGxeKuPU/3Pk3TtostPXAwOGce7lnTphwprTjfFulldW19Y3yZmVre2d3z94/aKs4lYS2SMxj2Q1BUc4EbWmmOe0mkkIUctoJx9e533mgUrFY3OtJQv0IhoINGAFtpMA+8iLQIwI8u51ij6QJhqAe2FWn5syAl4lbkCoq0AzsL68fkzSiQhMOSvVcJ9F+BlIzwum04qWKJkDGMKQ9QwVEVPnZLP0UnxqljwexNE9oPFN/b2QQKTWJQjOZZ1WLXi7+5/VSPaj7GRNJqqkg80ODlGMd47wK3GeSEs0nhgCRzGTFZAQSiDaFVUwJ7uKXl0n7vOY6Nffuotq4Kuooo2N0gs6Qiy5RA92gJmohgh7RM3pFb9aT9WK9Wx/z0ZJV7ByiP7A+fwDUc5TMAAAB/XicbVDLSsNAFJ3UV62v+Ni5GSyCq5KIYJdFN+6sYB/QhHAznbZDJ5MwMxFqKP6KGxeKuPU/3Pk3TtostPXAwOGce7lnTphwprTjfFulldW19Y3yZmVre2d3z94/aKs4lYS2SMxj2Q1BUc4EbWmmOe0mkkIUctoJx9e533mgUrFY3OtJQv0IhoINGAFtpMA+8iLQIwI8u51ij6QJhqAe2FWn5syAl4lbkCoq0AzsL68fkzSiQhMOSvVcJ9F+BlIzwum04qWKJkDGMKQ9QwVEVPnZLP0UnxqljwexNE9oPFN/b2QQKTWJQjOZZ1WLXi7+5/VSPaj7GRNJqqkg80ODlGMd47wK3GeSEs0nhgCRzGTFZAQSiDaFVUwJ7uKXl0n7vOY6Nffuotq4Kuooo2N0gs6Qiy5RA92gJmohgh7RM3pFb9aT9WK9Wx/z0ZJV7ByiP7A+fwDUc5TM...AAAB7XicbVBNS8NAEJ34WetX1aOXYBE8lUQEPRa9eKxgP6ANZbPZtGs3u2F3Uiih/8GLB0W8+n+8+W/ctjlo64OBx3szzMwLU8ENet63s7a+sbm1Xdop7+7tHxxWjo5bRmWasiZVQulOSAwTXLImchSsk2pGklCwdji6m/ntMdOGK/mIk5QFCRlIHnNK0Eqt3jhSaPqVqlfz5nBXiV+QKhRo9CtfvUjRLGESqSDGdH0vxSAnGjkVbFruZYalhI7IgHUtlSRhJsjn107dc6tEbqy0LYnuXP09kZPEmEkS2s6E4NAsezPxP6+bYXwT5FymGTJJF4viTLio3NnrbsQ1oygmlhCqub3VpUOiCUUbUNmG4C+/vEpalzXfq/kPV9X6bRFHCU7hDC7Ah2uowz00oAkUnuAZXuHNUc6L8+58LFrXnGLmBP7A+fwBy2+PQg==AAAB7XicbVBNS8NAEJ34WetX1aOXYBE8lUQEPRa9eKxgP6ANZbPZtGs3u2F3Uiih/8GLB0W8+n+8+W/ctjlo64OBx3szzMwLU8ENet63s7a+sbm1Xdop7+7tHxxWjo5bRmWasiZVQulOSAwTXLImchSsk2pGklCwdji6m/ntMdOGK/mIk5QFCRlIHnNK0Eqt3jhSaPqVqlfz5nBXiV+QKhRo9CtfvUjRLGESqSDGdH0vxSAnGjkVbFruZYalhI7IgHUtlSRhJsjn107dc6tEbqy0LYnuXP09kZPEmEkS2s6E4NAsezPxP6+bYXwT5FymGTJJF4viTLio3NnrbsQ1oygmlhCqub3VpUOiCUUbUNmG4C+/vEpalzXfq/kPV9X6bRFHCU7hDC7Ah2uowz00oAkUnuAZXuHNUc6L8+58LFrXnGLmBP7A+fwBy2+PQg==AAAB7XicbVBNS8NAEJ34WetX1aOXYBE8lUQEPRa9eKxgP6ANZbPZtGs3u2F3Uiih/8GLB0W8+n+8+W/ctjlo64OBx3szzMwLU8ENet63s7a+sbm1Xdop7+7tHxxWjo5bRmWasiZVQulOSAwTXLImchSsk2pGklCwdji6m/ntMdOGK/mIk5QFCRlIHnNK0Eqt3jhSaPqVqlfz5nBXiV+QKhRo9CtfvUjRLGESqSDGdH0vxSAnGjkVbFruZYalhI7IgHUtlSRhJsjn107dc6tEbqy0LYnuXP09kZPEmEkS2s6E4NAsezPxP6+bYXwT5FymGTJJF4viTLio3NnrbsQ1oygmlhCqub3VpUOiCUUbUNmG4C+/vEpalzXfq/kPV9X6bRFHCU7hDC7Ah2uowz00oAkUnuAZXuHNUc6L8+58LFrXnGLmBP7A+fwBy2+PQg==AAAB7XicbVBNS8NAEJ34WetX1aOXYBE8lUQEPRa9eKxgP6ANZbPZtGs3u2F3Uiih/8GLB0W8+n+8+W/ctjlo64OBx3szzMwLU8ENet63s7a+sbm1Xdop7+7tHxxWjo5bRmWasiZVQulOSAwTXLImchSsk2pGklCwdji6m/ntMdOGK/mIk5QFCRlIHnNK0Eqt3jhSaPqVqlfz5nBXiV+QKhRo9CtfvUjRLGESqSDGdH0vxSAnGjkVbFruZYalhI7IgHUtlSRhJsjn107dc6tEbqy0LYnuXP09kZPEmEkS2s6E4NAsezPxP6+bYXwT5FymGTJJF4viTLio3NnrbsQ1oygmlhCqub3VpUOiCUUbUNmG4C+/vEpalzXfq/kPV9X6bRFHCU7hDC7Ah2uowz00oAkUnuAZXuHNUc6L8+58LFrXnGLmBP7A+fwBy2+PQg==...AAAB7XicbVBNS8NAEJ34WetX1aOXYBE8lUQEPRa9eKxgP6ANZbPZtGs3u2F3Uiih/8GLB0W8+n+8+W/ctjlo64OBx3szzMwLU8ENet63s7a+sbm1Xdop7+7tHxxWjo5bRmWasiZVQulOSAwTXLImchSsk2pGklCwdji6m/ntMdOGK/mIk5QFCRlIHnNK0Eqt3jhSaPqVqlfz5nBXiV+QKhRo9CtfvUjRLGESqSDGdH0vxSAnGjkVbFruZYalhI7IgHUtlSRhJsjn107dc6tEbqy0LYnuXP09kZPEmEkS2s6E4NAsezPxP6+bYXwT5FymGTJJF4viTLio3NnrbsQ1oygmlhCqub3VpUOiCUUbUNmG4C+/vEpalzXfq/kPV9X6bRFHCU7hDC7Ah2uowz00oAkUnuAZXuHNUc6L8+58LFrXnGLmBP7A+fwBy2+PQg==AAAB7XicbVBNS8NAEJ34WetX1aOXYBE8lUQEPRa9eKxgP6ANZbPZtGs3u2F3Uiih/8GLB0W8+n+8+W/ctjlo64OBx3szzMwLU8ENet63s7a+sbm1Xdop7+7tHxxWjo5bRmWasiZVQulOSAwTXLImchSsk2pGklCwdji6m/ntMdOGK/mIk5QFCRlIHnNK0Eqt3jhSaPqVqlfz5nBXiV+QKhRo9CtfvUjRLGESqSDGdH0vxSAnGjkVbFruZYalhI7IgHUtlSRhJsjn107dc6tEbqy0LYnuXP09kZPEmEkS2s6E4NAsezPxP6+bYXwT5FymGTJJF4viTLio3NnrbsQ1oygmlhCqub3VpUOiCUUbUNmG4C+/vEpalzXfq/kPV9X6bRFHCU7hDC7Ah2uowz00oAkUnuAZXuHNUc6L8+58LFrXnGLmBP7A+fwBy2+PQg==AAAB7XicbVBNS8NAEJ34WetX1aOXYBE8lUQEPRa9eKxgP6ANZbPZtGs3u2F3Uiih/8GLB0W8+n+8+W/ctjlo64OBx3szzMwLU8ENet63s7a+sbm1Xdop7+7tHxxWjo5bRmWasiZVQulOSAwTXLImchSsk2pGklCwdji6m/ntMdOGK/mIk5QFCRlIHnNK0Eqt3jhSaPqVqlfz5nBXiV+QKhRo9CtfvUjRLGESqSDGdH0vxSAnGjkVbFruZYalhI7IgHUtlSRhJsjn107dc6tEbqy0LYnuXP09kZPEmEkS2s6E4NAsezPxP6+bYXwT5FymGTJJF4viTLio3NnrbsQ1oygmlhCqub3VpUOiCUUbUNmG4C+/vEpalzXfq/kPV9X6bRFHCU7hDC7Ah2uowz00oAkUnuAZXuHNUc6L8+58LFrXnGLmBP7A+fwBy2+PQg==AAAB7XicbVBNS8NAEJ34WetX1aOXYBE8lUQEPRa9eKxgP6ANZbPZtGs3u2F3Uiih/8GLB0W8+n+8+W/ctjlo64OBx3szzMwLU8ENet63s7a+sbm1Xdop7+7tHxxWjo5bRmWasiZVQulOSAwTXLImchSsk2pGklCwdji6m/ntMdOGK/mIk5QFCRlIHnNK0Eqt3jhSaPqVqlfz5nBXiV+QKhRo9CtfvUjRLGESqSDGdH0vxSAnGjkVbFruZYalhI7IgHUtlSRhJsjn107dc6tEbqy0LYnuXP09kZPEmEkS2s6E4NAsezPxP6+bYXwT5FymGTJJF4viTLio3NnrbsQ1oygmlhCqub3VpUOiCUUbUNmG4C+/vEpalzXfq/kPV9X6bRFHCU7hDC7Ah2uowz00oAkUnuAZXuHNUc6L8+58LFrXnGLmBP7A+fwBy2+PQg==SamplingAAAB73icbVA9SwNBEJ3zM8avqKXNYhCswl0aLYM2lhHNByRH2NvsJUt2987dOSGE/AkbC0Vs/Tt2/hs3yRWa+GDg8d4MM/OiVAqLvv/tra1vbG5tF3aKu3v7B4elo+OmTTLDeIMlMjHtiFouheYNFCh5OzWcqkjyVjS6mfmtJ26sSPQDjlMeKjrQIhaMopPa91S5LXrQK5X9ij8HWSVBTsqQo94rfXX7CcsU18gktbYT+CmGE2pQMMmnxW5meUrZiA54x1FNFbfhZH7vlJw7pU/ixLjSSObq74kJVdaOVeQ6FcWhXfZm4n9eJ8P4KpwInWbINVssijNJMCGz50lfGM5Qjh2hzAh3K2FDaihDF1HRhRAsv7xKmtVK4FeCu2q5dp3HUYBTOIMLCOASanALdWgAAwnP8Apv3qP34r17H4vWNS+fOYE/8D5/ACLVkAM=AAAB73icbVA9SwNBEJ3zM8avqKXNYhCswl0aLYM2lhHNByRH2NvsJUt2987dOSGE/AkbC0Vs/Tt2/hs3yRWa+GDg8d4MM/OiVAqLvv/tra1vbG5tF3aKu3v7B4elo+OmTTLDeIMlMjHtiFouheYNFCh5OzWcqkjyVjS6mfmtJ26sSPQDjlMeKjrQIhaMopPa91S5LXrQK5X9ij8HWSVBTsqQo94rfXX7CcsU18gktbYT+CmGE2pQMMmnxW5meUrZiA54x1FNFbfhZH7vlJw7pU/ixLjSSObq74kJVdaOVeQ6FcWhXfZm4n9eJ8P4KpwInWbINVssijNJMCGz50lfGM5Qjh2hzAh3K2FDaihDF1HRhRAsv7xKmtVK4FeCu2q5dp3HUYBTOIMLCOASanALdWgAAwnP8Apv3qP34r17H4vWNS+fOYE/8D5/ACLVkAM=AAAB73icbVA9SwNBEJ3zM8avqKXNYhCswl0aLYM2lhHNByRH2NvsJUt2987dOSGE/AkbC0Vs/Tt2/hs3yRWa+GDg8d4MM/OiVAqLvv/tra1vbG5tF3aKu3v7B4elo+OmTTLDeIMlMjHtiFouheYNFCh5OzWcqkjyVjS6mfmtJ26sSPQDjlMeKjrQIhaMopPa91S5LXrQK5X9ij8HWSVBTsqQo94rfXX7CcsU18gktbYT+CmGE2pQMMmnxW5meUrZiA54x1FNFbfhZH7vlJw7pU/ixLjSSObq74kJVdaOVeQ6FcWhXfZm4n9eJ8P4KpwInWbINVssijNJMCGz50lfGM5Qjh2hzAh3K2FDaihDF1HRhRAsv7xKmtVK4FeCu2q5dp3HUYBTOIMLCOASanALdWgAAwnP8Apv3qP34r17H4vWNS+fOYE/8D5/ACLVkAM=AAAB73icbVA9SwNBEJ3zM8avqKXNYhCswl0aLYM2lhHNByRH2NvsJUt2987dOSGE/AkbC0Vs/Tt2/hs3yRWa+GDg8d4MM/OiVAqLvv/tra1vbG5tF3aKu3v7B4elo+OmTTLDeIMlMjHtiFouheYNFCh5OzWcqkjyVjS6mfmtJ26sSPQDjlMeKjrQIhaMopPa91S5LXrQK5X9ij8HWSVBTsqQo94rfXX7CcsU18gktbYT+CmGE2pQMMmnxW5meUrZiA54x1FNFbfhZH7vlJw7pU/ixLjSSObq74kJVdaOVeQ6FcWhXfZm4n9eJ8P4KpwInWbINVssijNJMCGz50lfGM5Qjh2hzAh3K2FDaihDF1HRhRAsv7xKmtVK4FeCu2q5dp3HUYBTOIMLCOASanALdWgAAwnP8Apv3qP34r17H4vWNS+fOYE/8D5/ACLVkAM=ResBlockAAAB73icbVA9TwJBEJ3DL8Qv1NLmIjGxInc0WhJsLNHIRwIXsrfMwYa93XN3z4Rc+BM2Fhpj69+x89+4wBUKvmSSl/dmMjMvTDjTxvO+ncLG5tb2TnG3tLd/cHhUPj5pa5kqii0quVTdkGjkTGDLMMOxmygkccixE05u5n7nCZVmUjyYaYJBTEaCRYwSY6XuPeoGl3QyKFe8qreAu078nFQgR3NQ/uoPJU1jFIZyonXP9xITZEQZRjnOSv1UY0LohIywZ6kgMeogW9w7cy+sMnQjqWwJ4y7U3xMZibWexqHtjIkZ61VvLv7n9VITXQcZE0lqUNDloijlrpHu/Hl3yBRSw6eWEKqYvdWlY6IINTaikg3BX315nbRrVd+r+ne1Sr2Rx1GEMziHS/DhCupwC01oAQUOz/AKb86j8+K8Ox/L1oKTz5zCHzifP+imj90=AAAB73icbVA9TwJBEJ3DL8Qv1NLmIjGxInc0WhJsLNHIRwIXsrfMwYa93XN3z4Rc+BM2Fhpj69+x89+4wBUKvmSSl/dmMjMvTDjTxvO+ncLG5tb2TnG3tLd/cHhUPj5pa5kqii0quVTdkGjkTGDLMMOxmygkccixE05u5n7nCZVmUjyYaYJBTEaCRYwSY6XuPeoGl3QyKFe8qreAu078nFQgR3NQ/uoPJU1jFIZyonXP9xITZEQZRjnOSv1UY0LohIywZ6kgMeogW9w7cy+sMnQjqWwJ4y7U3xMZibWexqHtjIkZ61VvLv7n9VITXQcZE0lqUNDloijlrpHu/Hl3yBRSw6eWEKqYvdWlY6IINTaikg3BX315nbRrVd+r+ne1Sr2Rx1GEMziHS/DhCupwC01oAQUOz/AKb86j8+K8Ox/L1oKTz5zCHzifP+imj90=AAAB73icbVA9TwJBEJ3DL8Qv1NLmIjGxInc0WhJsLNHIRwIXsrfMwYa93XN3z4Rc+BM2Fhpj69+x89+4wBUKvmSSl/dmMjMvTDjTxvO+ncLG5tb2TnG3tLd/cHhUPj5pa5kqii0quVTdkGjkTGDLMMOxmygkccixE05u5n7nCZVmUjyYaYJBTEaCRYwSY6XuPeoGl3QyKFe8qreAu078nFQgR3NQ/uoPJU1jFIZyonXP9xITZEQZRjnOSv1UY0LohIywZ6kgMeogW9w7cy+sMnQjqWwJ4y7U3xMZibWexqHtjIkZ61VvLv7n9VITXQcZE0lqUNDloijlrpHu/Hl3yBRSw6eWEKqYvdWlY6IINTaikg3BX315nbRrVd+r+ne1Sr2Rx1GEMziHS/DhCupwC01oAQUOz/AKb86j8+K8Ox/L1oKTz5zCHzifP+imj90=AAAB73icbVA9TwJBEJ3DL8Qv1NLmIjGxInc0WhJsLNHIRwIXsrfMwYa93XN3z4Rc+BM2Fhpj69+x89+4wBUKvmSSl/dmMjMvTDjTxvO+ncLG5tb2TnG3tLd/cHhUPj5pa5kqii0quVTdkGjkTGDLMMOxmygkccixE05u5n7nCZVmUjyYaYJBTEaCRYwSY6XuPeoGl3QyKFe8qreAu078nFQgR3NQ/uoPJU1jFIZyonXP9xITZEQZRjnOSv1UY0LohIywZ6kgMeogW9w7cy+sMnQjqWwJ4y7U3xMZibWexqHtjIkZ61VvLv7n9VITXQcZE0lqUNDloijlrpHu/Hl3yBRSw6eWEKqYvdWlY6IINTaikg3BX315nbRrVd+r+ne1Sr2Rx1GEMziHS/DhCupwC01oAQUOz/AKb86j8+K8Ox/L1oKTz5zCHzifP+imj90=MLPAAAB6nicbVC7SgNBFL0bXzG+opY2g0GwCrtpTBm0sVCIaB6QLDI7uZsMmZ1dZmaFsOQTbCwUsfWL7PwbJ8kWmnhg4HDOucy9J0gE18Z1v53C2vrG5lZxu7Szu7d/UD48aus4VQxbLBax6gZUo+ASW4Ybgd1EIY0CgZ1gfDXzO0+oNI/lg5kk6Ed0KHnIGTVWur+9aT6WK27VnYOsEi8nFchh81/9QczSCKVhgmrd89zE+BlVhjOB01I/1ZhQNqZD7FkqaYTaz+arTsmZVQYkjJV90pC5+nsio5HWkyiwyYiakV72ZuJ/Xi81Yd3PuExSg5ItPgpTQUxMZneTAVfIjJhYQpnidlfCRlRRZmw7JVuCt3zyKmnXqp5b9e5qlcZlXkcRTuAUzsGDC2jANTShBQyG8Ayv8OYI58V5dz4W0YKTzxzDHzifP9ucjX8=AAAB6nicbVC7SgNBFL0bXzG+opY2g0GwCrtpTBm0sVCIaB6QLDI7uZsMmZ1dZmaFsOQTbCwUsfWL7PwbJ8kWmnhg4HDOucy9J0gE18Z1v53C2vrG5lZxu7Szu7d/UD48aus4VQxbLBax6gZUo+ASW4Ybgd1EIY0CgZ1gfDXzO0+oNI/lg5kk6Ed0KHnIGTVWur+9aT6WK27VnYOsEi8nFchh81/9QczSCKVhgmrd89zE+BlVhjOB01I/1ZhQNqZD7FkqaYTaz+arTsmZVQYkjJV90pC5+nsio5HWkyiwyYiakV72ZuJ/Xi81Yd3PuExSg5ItPgpTQUxMZneTAVfIjJhYQpnidlfCRlRRZmw7JVuCt3zyKmnXqp5b9e5qlcZlXkcRTuAUzsGDC2jANTShBQyG8Ayv8OYI58V5dz4W0YKTzxzDHzifP9ucjX8=AAAB6nicbVC7SgNBFL0bXzG+opY2g0GwCrtpTBm0sVCIaB6QLDI7uZsMmZ1dZmaFsOQTbCwUsfWL7PwbJ8kWmnhg4HDOucy9J0gE18Z1v53C2vrG5lZxu7Szu7d/UD48aus4VQxbLBax6gZUo+ASW4Ybgd1EIY0CgZ1gfDXzO0+oNI/lg5kk6Ed0KHnIGTVWur+9aT6WK27VnYOsEi8nFchh81/9QczSCKVhgmrd89zE+BlVhjOB01I/1ZhQNqZD7FkqaYTaz+arTsmZVQYkjJV90pC5+nsio5HWkyiwyYiakV72ZuJ/Xi81Yd3PuExSg5ItPgpTQUxMZneTAVfIjJhYQpnidlfCRlRRZmw7JVuCt3zyKmnXqp5b9e5qlcZlXkcRTuAUzsGDC2jANTShBQyG8Ayv8OYI58V5dz4W0YKTzxzDHzifP9ucjX8=AAAB6nicbVC7SgNBFL0bXzG+opY2g0GwCrtpTBm0sVCIaB6QLDI7uZsMmZ1dZmaFsOQTbCwUsfWL7PwbJ8kWmnhg4HDOucy9J0gE18Z1v53C2vrG5lZxu7Szu7d/UD48aus4VQxbLBax6gZUo+ASW4Ybgd1EIY0CgZ1gfDXzO0+oNI/lg5kk6Ed0KHnIGTVWur+9aT6WK27VnYOsEi8nFchh81/9QczSCKVhgmrd89zE+BlVhjOB01I/1ZhQNqZD7FkqaYTaz+arTsmZVQYkjJV90pC5+nsio5HWkyiwyYiakV72ZuJ/Xi81Yd3PuExSg5ItPgpTQUxMZneTAVfIjJhYQpnidlfCRlRRZmw7JVuCt3zyKmnXqp5b9e5qlcZlXkcRTuAUzsGDC2jANTShBQyG8Ayv8OYI58V5dz4W0YKTzxzDHzifP9ucjX8=ContrastModuleAAAB+HicbVBNS8NAEN34WetHox69LBbBU0l60WOxFy9CBfsBbSibzaZdutkNuxOhhv4SLx4U8epP8ea/cdvmoK0PBh7vzTAzL0wFN+B5387G5tb2zm5pr7x/cHhUcY9POkZlmrI2VULpXkgME1yyNnAQrJdqRpJQsG44ac797iPThiv5ANOUBQkZSR5zSsBKQ7fSVBI0MYDvVJQJNnSrXs1bAK8TvyBVVKA1dL8GkaJZwiRQQYzp+14KQU40cCrYrDzIDEsJnZAR61sqScJMkC8On+ELq0Q4VtqWBLxQf0/kJDFmmoS2MyEwNqveXPzP62cQXwc5l2kGTNLlojgTGBSep4AjrhkFMbWEUM3trZiOiSYUbFZlG4K/+vI66dRrvlfz7+vVxk0RRwmdoXN0iXx0hRroFrVQG1GUoWf0it6cJ+fFeXc+lq0bTjFziv7A+fwBqyKTEw==AAAB+HicbVBNS8NAEN34WetHox69LBbBU0l60WOxFy9CBfsBbSibzaZdutkNuxOhhv4SLx4U8epP8ea/cdvmoK0PBh7vzTAzL0wFN+B5387G5tb2zm5pr7x/cHhUcY9POkZlmrI2VULpXkgME1yyNnAQrJdqRpJQsG44ac797iPThiv5ANOUBQkZSR5zSsBKQ7fSVBI0MYDvVJQJNnSrXs1bAK8TvyBVVKA1dL8GkaJZwiRQQYzp+14KQU40cCrYrDzIDEsJnZAR61sqScJMkC8On+ELq0Q4VtqWBLxQf0/kJDFmmoS2MyEwNqveXPzP62cQXwc5l2kGTNLlojgTGBSep4AjrhkFMbWEUM3trZiOiSYUbFZlG4K/+vI66dRrvlfz7+vVxk0RRwmdoXN0iXx0hRroFrVQG1GUoWf0it6cJ+fFeXc+lq0bTjFziv7A+fwBqyKTEw==AAAB+HicbVBNS8NAEN34WetHox69LBbBU0l60WOxFy9CBfsBbSibzaZdutkNuxOhhv4SLx4U8epP8ea/cdvmoK0PBh7vzTAzL0wFN+B5387G5tb2zm5pr7x/cHhUcY9POkZlmrI2VULpXkgME1yyNnAQrJdqRpJQsG44ac797iPThiv5ANOUBQkZSR5zSsBKQ7fSVBI0MYDvVJQJNnSrXs1bAK8TvyBVVKA1dL8GkaJZwiRQQYzp+14KQU40cCrYrDzIDEsJnZAR61sqScJMkC8On+ELq0Q4VtqWBLxQf0/kJDFmmoS2MyEwNqveXPzP62cQXwc5l2kGTNLlojgTGBSep4AjrhkFMbWEUM3trZiOiSYUbFZlG4K/+vI66dRrvlfz7+vVxk0RRwmdoXN0iXx0hRroFrVQG1GUoWf0it6cJ+fFeXc+lq0bTjFziv7A+fwBqyKTEw==AAAB+HicbVBNS8NAEN34WetHox69LBbBU0l60WOxFy9CBfsBbSibzaZdutkNuxOhhv4SLx4U8epP8ea/cdvmoK0PBh7vzTAzL0wFN+B5387G5tb2zm5pr7x/cHhUcY9POkZlmrI2VULpXkgME1yyNnAQrJdqRpJQsG44ac797iPThiv5ANOUBQkZSR5zSsBKQ7fSVBI0MYDvVJQJNnSrXs1bAK8TvyBVVKA1dL8GkaJZwiRQQYzp+14KQU40cCrYrDzIDEsJnZAR61sqScJMkC8On+ELq0Q4VtqWBLxQf0/kJDFmmoS2MyEwNqveXPzP62cQXwc5l2kGTNLlojgTGBSep4AjrhkFMbWEUM3trZiOiSYUbFZlG4K/+vI66dRrvlfz7+vVxk0RRwmdoXN0iXx0hRroFrVQG1GUoWf0it6cJ+fFeXc+lq0bTjFziv7A+fwBqyKTEw==ContrastLossAAAB9HicbVA9TwJBEJ3zE/ELtbTZSEysyB2NlkQaCwtM5COBC9lbFtiwt3vuzpEQwu+wsdAYW3+Mnf/GBa5Q8CWTvLw3k5l5USKFRd//9jY2t7Z3dnN7+f2Dw6Pjwslpw+rUMF5nWmrTiqjlUiheR4GStxLDaRxJ3oxG1bnfHHNjhVaPOEl4GNOBEn3BKDoprGqFhlok99rabqHol/wFyDoJMlKEDLVu4avT0yyNuUImqbXtwE8wnFKDgkk+y3dSyxPKRnTA244qGnMbThdHz8ilU3qkr40rhWSh/p6Y0tjaSRy5zpji0K56c/E/r51i/yacCpWkyBVbLuqnkqAm8wRITxjOUE4cocwIdythQ2ooQ5dT3oUQrL68ThrlUuCXgodysXKbxZGDc7iAKwjgGipwBzWoA4MneIZXePPG3ov37n0sWze8bOYM/sD7/AG0W5IJAAAB9HicbVA9TwJBEJ3zE/ELtbTZSEysyB2NlkQaCwtM5COBC9lbFtiwt3vuzpEQwu+wsdAYW3+Mnf/GBa5Q8CWTvLw3k5l5USKFRd//9jY2t7Z3dnN7+f2Dw6Pjwslpw+rUMF5nWmrTiqjlUiheR4GStxLDaRxJ3oxG1bnfHHNjhVaPOEl4GNOBEn3BKDoprGqFhlok99rabqHol/wFyDoJMlKEDLVu4avT0yyNuUImqbXtwE8wnFKDgkk+y3dSyxPKRnTA244qGnMbThdHz8ilU3qkr40rhWSh/p6Y0tjaSRy5zpji0K56c/E/r51i/yacCpWkyBVbLuqnkqAm8wRITxjOUE4cocwIdythQ2ooQ5dT3oUQrL68ThrlUuCXgodysXKbxZGDc7iAKwjgGipwBzWoA4MneIZXePPG3ov37n0sWze8bOYM/sD7/AG0W5IJAAAB9HicbVA9TwJBEJ3zE/ELtbTZSEysyB2NlkQaCwtM5COBC9lbFtiwt3vuzpEQwu+wsdAYW3+Mnf/GBa5Q8CWTvLw3k5l5USKFRd//9jY2t7Z3dnN7+f2Dw6Pjwslpw+rUMF5nWmrTiqjlUiheR4GStxLDaRxJ3oxG1bnfHHNjhVaPOEl4GNOBEn3BKDoprGqFhlok99rabqHol/wFyDoJMlKEDLVu4avT0yyNuUImqbXtwE8wnFKDgkk+y3dSyxPKRnTA244qGnMbThdHz8ilU3qkr40rhWSh/p6Y0tjaSRy5zpji0K56c/E/r51i/yacCpWkyBVbLuqnkqAm8wRITxjOUE4cocwIdythQ2ooQ5dT3oUQrL68ThrlUuCXgodysXKbxZGDc7iAKwjgGipwBzWoA4MneIZXePPG3ov37n0sWze8bOYM/sD7/AG0W5IJAAAB9HicbVA9TwJBEJ3zE/ELtbTZSEysyB2NlkQaCwtM5COBC9lbFtiwt3vuzpEQwu+wsdAYW3+Mnf/GBa5Q8CWTvLw3k5l5USKFRd//9jY2t7Z3dnN7+f2Dw6Pjwslpw+rUMF5nWmrTiqjlUiheR4GStxLDaRxJ3oxG1bnfHHNjhVaPOEl4GNOBEn3BKDoprGqFhlok99rabqHol/wFyDoJMlKEDLVu4avT0yyNuUImqbXtwE8wnFKDgkk+y3dSyxPKRnTA244qGnMbThdHz8ilU3qkr40rhWSh/p6Y0tjaSRy5zpji0K56c/E/r51i/yacCpWkyBVbLuqnkqAm8wRITxjOUE4cocwIdythQ2ooQ5dT3oUQrL68ThrlUuCXgodysXKbxZGDc7iAKwjgGipwBzWoA4MneIZXePPG3ov37n0sWze8bOYM/sD7/AG0W5IJInferenceBranchAAAB+XicbVA9SwNBEJ2LXzF+nVraLAbBKtyl0TLERrsI5gOSEPY2c8mSvb1jdy8QjvwTGwtFbP0ndv4bN8kVmvhg4PHeDDPzgkRwbTzv2ylsbe/s7hX3SweHR8cn7ulZS8epYthksYhVJ6AaBZfYNNwI7CQKaRQIbAeTu4XfnqLSPJZPZpZgP6IjyUPOqLHSwHUfZIgKJUNSV1Sy8cAtexVvCbJJ/JyUIUdj4H71hjFLI5SGCap11/cS08+oMpwJnJd6qcaEsgkdYddSSSPU/Wx5+ZxcWWVIwljZkoYs1d8TGY20nkWB7YyoGet1byH+53VTE972My6T1NjnVovCVBATk0UMZMgVMiNmllCmuL2VsDFVlBkbVsmG4K+/vEla1YrvVfzHarlWz+MowgVcwjX4cAM1uIcGNIHBFJ7hFd6czHlx3p2PVWvByWfO4Q+czx8S3ZNGAAAB+XicbVA9SwNBEJ2LXzF+nVraLAbBKtyl0TLERrsI5gOSEPY2c8mSvb1jdy8QjvwTGwtFbP0ndv4bN8kVmvhg4PHeDDPzgkRwbTzv2ylsbe/s7hX3SweHR8cn7ulZS8epYthksYhVJ6AaBZfYNNwI7CQKaRQIbAeTu4XfnqLSPJZPZpZgP6IjyUPOqLHSwHUfZIgKJUNSV1Sy8cAtexVvCbJJ/JyUIUdj4H71hjFLI5SGCap11/cS08+oMpwJnJd6qcaEsgkdYddSSSPU/Wx5+ZxcWWVIwljZkoYs1d8TGY20nkWB7YyoGet1byH+53VTE972My6T1NjnVovCVBATk0UMZMgVMiNmllCmuL2VsDFVlBkbVsmG4K+/vEla1YrvVfzHarlWz+MowgVcwjX4cAM1uIcGNIHBFJ7hFd6czHlx3p2PVWvByWfO4Q+czx8S3ZNGAAAB+XicbVA9SwNBEJ2LXzF+nVraLAbBKtyl0TLERrsI5gOSEPY2c8mSvb1jdy8QjvwTGwtFbP0ndv4bN8kVmvhg4PHeDDPzgkRwbTzv2ylsbe/s7hX3SweHR8cn7ulZS8epYthksYhVJ6AaBZfYNNwI7CQKaRQIbAeTu4XfnqLSPJZPZpZgP6IjyUPOqLHSwHUfZIgKJUNSV1Sy8cAtexVvCbJJ/JyUIUdj4H71hjFLI5SGCap11/cS08+oMpwJnJd6qcaEsgkdYddSSSPU/Wx5+ZxcWWVIwljZkoYs1d8TGY20nkWB7YyoGet1byH+53VTE972My6T1NjnVovCVBATk0UMZMgVMiNmllCmuL2VsDFVlBkbVsmG4K+/vEla1YrvVfzHarlWz+MowgVcwjX4cAM1uIcGNIHBFJ7hFd6czHlx3p2PVWvByWfO4Q+czx8S3ZNGAAAB+XicbVA9SwNBEJ2LXzF+nVraLAbBKtyl0TLERrsI5gOSEPY2c8mSvb1jdy8QjvwTGwtFbP0ndv4bN8kVmvhg4PHeDDPzgkRwbTzv2ylsbe/s7hX3SweHR8cn7ulZS8epYthksYhVJ6AaBZfYNNwI7CQKaRQIbAeTu4XfnqLSPJZPZpZgP6IjyUPOqLHSwHUfZIgKJUNSV1Sy8cAtexVvCbJJ/JyUIUdj4H71hjFLI5SGCap11/cS08+oMpwJnJd6qcaEsgkdYddSSSPU/Wx5+ZxcWWVIwljZkoYs1d8TGY20nkWB7YyoGet1byH+53VTE972My6T1NjnVovCVBATk0UMZMgVMiNmllCmuL2VsDFVlBkbVsmG4K+/vEla1YrvVfzHarlWz+MowgVcwjX4cAM1uIcGNIHBFJ7hFd6czHlx3p2PVWvByWfO4Q+czx8S3ZNGPerceptionBranchAAAB+nicbVDLTgIxFL3jE/EFunTTSExckRk2uiS4cYmJPBIgpFPuQEOnM2k7GjLyKW5caIxbv8Sdf2MHZqHgSZqcnnPv7e3xY8G1cd1vZ2Nza3tnt7BX3D84PDoulU/aOkoUwxaLRKS6PtUouMSW4UZgN1ZIQ19gx5/eZH7nAZXmkbw3sxgHIR1LHnBGjZWGpXIT7aA4u5CGopJNhqWKW3UXIOvEy0kFcjSHpa/+KGJJiNIwQbXueW5sBilVhjOB82I/0RhTNqVj7FkqaYh6kC5Wn5MLq4xIECl7pCEL9XdHSkOtZ6FvK0NqJnrVy8T/vF5igutBymWcGJRs+VCQCGIikuVARlwhM2JmCWWK210Jm1BFmbFpFW0I3uqX10m7VvXcqndXq9QbeRwFOINzuAQPrqAOt9CEFjB4hGd4hTfnyXlx3p2PZemGk/ecwh84nz8Q3JPaAAAB+nicbVDLTgIxFL3jE/EFunTTSExckRk2uiS4cYmJPBIgpFPuQEOnM2k7GjLyKW5caIxbv8Sdf2MHZqHgSZqcnnPv7e3xY8G1cd1vZ2Nza3tnt7BX3D84PDoulU/aOkoUwxaLRKS6PtUouMSW4UZgN1ZIQ19gx5/eZH7nAZXmkbw3sxgHIR1LHnBGjZWGpXIT7aA4u5CGopJNhqWKW3UXIOvEy0kFcjSHpa/+KGJJiNIwQbXueW5sBilVhjOB82I/0RhTNqVj7FkqaYh6kC5Wn5MLq4xIECl7pCEL9XdHSkOtZ6FvK0NqJnrVy8T/vF5igutBymWcGJRs+VCQCGIikuVARlwhM2JmCWWK210Jm1BFmbFpFW0I3uqX10m7VvXcqndXq9QbeRwFOINzuAQPrqAOt9CEFjB4hGd4hTfnyXlx3p2PZemGk/ecwh84nz8Q3JPaAAAB+nicbVDLTgIxFL3jE/EFunTTSExckRk2uiS4cYmJPBIgpFPuQEOnM2k7GjLyKW5caIxbv8Sdf2MHZqHgSZqcnnPv7e3xY8G1cd1vZ2Nza3tnt7BX3D84PDoulU/aOkoUwxaLRKS6PtUouMSW4UZgN1ZIQ19gx5/eZH7nAZXmkbw3sxgHIR1LHnBGjZWGpXIT7aA4u5CGopJNhqWKW3UXIOvEy0kFcjSHpa/+KGJJiNIwQbXueW5sBilVhjOB82I/0RhTNqVj7FkqaYh6kC5Wn5MLq4xIECl7pCEL9XdHSkOtZ6FvK0NqJnrVy8T/vF5igutBymWcGJRs+VCQCGIikuVARlwhM2JmCWWK210Jm1BFmbFpFW0I3uqX10m7VvXcqndXq9QbeRwFOINzuAQPrqAOt9CEFjB4hGd4hTfnyXlx3p2PZemGk/ecwh84nz8Q3JPaAAAB+nicbVDLTgIxFL3jE/EFunTTSExckRk2uiS4cYmJPBIgpFPuQEOnM2k7GjLyKW5caIxbv8Sdf2MHZqHgSZqcnnPv7e3xY8G1cd1vZ2Nza3tnt7BX3D84PDoulU/aOkoUwxaLRKS6PtUouMSW4UZgN1ZIQ19gx5/eZH7nAZXmkbw3sxgHIR1LHnBGjZWGpXIT7aA4u5CGopJNhqWKW3UXIOvEy0kFcjSHpa/+KGJJiNIwQbXueW5sBilVhjOB82I/0RhTNqVj7FkqaYh6kC5Wn5MLq4xIECl7pCEL9XdHSkOtZ6FvK0NqJnrVy8T/vF5igutBymWcGJRs+VCQCGIikuVARlwhM2JmCWWK210Jm1BFmbFpFW0I3uqX10m7VvXcqndXq9QbeRwFOINzuAQPrqAOt9CEFjB4hGd4hTfnyXlx3p2PZemGk/ecwh84nz8Q3JPa...AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cK9gPaUDabTbt2kw27E6GE/gcvHhTx6v/x5r9x2+agrQ8GHu/NMDMvSKUw6LrfTmltfWNzq7xd2dnd2z+oHh61jco04y2mpNLdgBouRcJbKFDybqo5jQPJO8H4duZ3nrg2QiUPOEm5H9NhIiLBKFqp3R+FCs2gWnPr7hxklXgFqUGB5qD61Q8Vy2KeIJPUmJ7npujnVKNgkk8r/czwlLIxHfKepQmNufHz+bVTcmaVkERK20qQzNXfEzmNjZnEge2MKY7MsjcT//N6GUbXfi6SNEOesMWiKJMEFZm9TkKhOUM5sYQyLeythI2opgxtQBUbgrf88ippX9Q9t+7dX9YaN0UcZTiBUzgHD66gAXfQhBYweIRneIU3RzkvzrvzsWgtOcXMMfyB8/kDtf+PNA==AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cK9gPaUDabTbt2kw27E6GE/gcvHhTx6v/x5r9x2+agrQ8GHu/NMDMvSKUw6LrfTmltfWNzq7xd2dnd2z+oHh61jco04y2mpNLdgBouRcJbKFDybqo5jQPJO8H4duZ3nrg2QiUPOEm5H9NhIiLBKFqp3R+FCs2gWnPr7hxklXgFqUGB5qD61Q8Vy2KeIJPUmJ7npujnVKNgkk8r/czwlLIxHfKepQmNufHz+bVTcmaVkERK20qQzNXfEzmNjZnEge2MKY7MsjcT//N6GUbXfi6SNEOesMWiKJMEFZm9TkKhOUM5sYQyLeythI2opgxtQBUbgrf88ippX9Q9t+7dX9YaN0UcZTiBUzgHD66gAXfQhBYweIRneIU3RzkvzrvzsWgtOcXMMfyB8/kDtf+PNA==AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cK9gPaUDabTbt2kw27E6GE/gcvHhTx6v/x5r9x2+agrQ8GHu/NMDMvSKUw6LrfTmltfWNzq7xd2dnd2z+oHh61jco04y2mpNLdgBouRcJbKFDybqo5jQPJO8H4duZ3nrg2QiUPOEm5H9NhIiLBKFqp3R+FCs2gWnPr7hxklXgFqUGB5qD61Q8Vy2KeIJPUmJ7npujnVKNgkk8r/czwlLIxHfKepQmNufHz+bVTcmaVkERK20qQzNXfEzmNjZnEge2MKY7MsjcT//N6GUbXfi6SNEOesMWiKJMEFZm9TkKhOUM5sYQyLeythI2opgxtQBUbgrf88ippX9Q9t+7dX9YaN0UcZTiBUzgHD66gAXfQhBYweIRneIU3RzkvzrvzsWgtOcXMMfyB8/kDtf+PNA==AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cK9gPaUDabTbt2kw27E6GE/gcvHhTx6v/x5r9x2+agrQ8GHu/NMDMvSKUw6LrfTmltfWNzq7xd2dnd2z+oHh61jco04y2mpNLdgBouRcJbKFDybqo5jQPJO8H4duZ3nrg2QiUPOEm5H9NhIiLBKFqp3R+FCs2gWnPr7hxklXgFqUGB5qD61Q8Vy2KeIJPUmJ7npujnVKNgkk8r/czwlLIxHfKepQmNufHz+bVTcmaVkERK20qQzNXfEzmNjZnEge2MKY7MsjcT//N6GUbXfi6SNEOesMWiKJMEFZm9TkKhOUM5sYQyLeythI2opgxtQBUbgrf88ippX9Q9t+7dX9YaN0UcZTiBUzgHD66gAXfQhBYweIRneIU3RzkvzrvzsWgtOcXMMfyB8/kDtf+PNA==...AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cK9gPaUDabTbt2kw27E6GE/gcvHhTx6v/x5r9x2+agrQ8GHu/NMDMvSKUw6LrfTmltfWNzq7xd2dnd2z+oHh61jco04y2mpNLdgBouRcJbKFDybqo5jQPJO8H4duZ3nrg2QiUPOEm5H9NhIiLBKFqp3R+FCs2gWnPr7hxklXgFqUGB5qD61Q8Vy2KeIJPUmJ7npujnVKNgkk8r/czwlLIxHfKepQmNufHz+bVTcmaVkERK20qQzNXfEzmNjZnEge2MKY7MsjcT//N6GUbXfi6SNEOesMWiKJMEFZm9TkKhOUM5sYQyLeythI2opgxtQBUbgrf88ippX9Q9t+7dX9YaN0UcZTiBUzgHD66gAXfQhBYweIRneIU3RzkvzrvzsWgtOcXMMfyB8/kDtf+PNA==AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cK9gPaUDabTbt2kw27E6GE/gcvHhTx6v/x5r9x2+agrQ8GHu/NMDMvSKUw6LrfTmltfWNzq7xd2dnd2z+oHh61jco04y2mpNLdgBouRcJbKFDybqo5jQPJO8H4duZ3nrg2QiUPOEm5H9NhIiLBKFqp3R+FCs2gWnPr7hxklXgFqUGB5qD61Q8Vy2KeIJPUmJ7npujnVKNgkk8r/czwlLIxHfKepQmNufHz+bVTcmaVkERK20qQzNXfEzmNjZnEge2MKY7MsjcT//N6GUbXfi6SNEOesMWiKJMEFZm9TkKhOUM5sYQyLeythI2opgxtQBUbgrf88ippX9Q9t+7dX9YaN0UcZTiBUzgHD66gAXfQhBYweIRneIU3RzkvzrvzsWgtOcXMMfyB8/kDtf+PNA==AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cK9gPaUDabTbt2kw27E6GE/gcvHhTx6v/x5r9x2+agrQ8GHu/NMDMvSKUw6LrfTmltfWNzq7xd2dnd2z+oHh61jco04y2mpNLdgBouRcJbKFDybqo5jQPJO8H4duZ3nrg2QiUPOEm5H9NhIiLBKFqp3R+FCs2gWnPr7hxklXgFqUGB5qD61Q8Vy2KeIJPUmJ7npujnVKNgkk8r/czwlLIxHfKepQmNufHz+bVTcmaVkERK20qQzNXfEzmNjZnEge2MKY7MsjcT//N6GUbXfi6SNEOesMWiKJMEFZm9TkKhOUM5sYQyLeythI2opgxtQBUbgrf88ippX9Q9t+7dX9YaN0UcZTiBUzgHD66gAXfQhBYweIRneIU3RzkvzrvzsWgtOcXMMfyB8/kDtf+PNA==AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cK9gPaUDabTbt2kw27E6GE/gcvHhTx6v/x5r9x2+agrQ8GHu/NMDMvSKUw6LrfTmltfWNzq7xd2dnd2z+oHh61jco04y2mpNLdgBouRcJbKFDybqo5jQPJO8H4duZ3nrg2QiUPOEm5H9NhIiLBKFqp3R+FCs2gWnPr7hxklXgFqUGB5qD61Q8Vy2KeIJPUmJ7npujnVKNgkk8r/czwlLIxHfKepQmNufHz+bVTcmaVkERK20qQzNXfEzmNjZnEge2MKY7MsjcT//N6GUbXfi6SNEOesMWiKJMEFZm9TkKhOUM5sYQyLeythI2opgxtQBUbgrf88ippX9Q9t+7dX9YaN0UcZTiBUzgHD66gAXfQhBYweIRneIU3RzkvzrvzsWgtOcXMMfyB8/kDtf+PNA==EncoderAAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4Kkk96LEogscK9gPaUDabSbt0swm7G6GE/ggvHhTx6u/x5r9x2+agrQ8GHu/NMDMvSAXXxnW/nbX1jc2t7dJOeXdv/+CwcnTc1kmmGLZYIhLVDahGwSW2DDcCu6lCGgcCO8H4duZ3nlBpnshHM0nRj+lQ8ogzaqzUuZMsCVENKlW35s5BVolXkCoUaA4qX/0wYVmM0jBBte55bmr8nCrDmcBpuZ9pTCkb0yH2LJU0Ru3n83On5NwqIYkSZUsaMld/T+Q01noSB7Yzpmakl72Z+J/Xy0x07edcpplByRaLokwQk5DZ7yTkCpkRE0soU9zeStiIKsqMTahsQ/CWX14l7XrNu6zVH+rVxk0RRwlO4QwuwIMraMA9NKEFDMbwDK/w5qTOi/PufCxa15xi5gT+wPn8AUGSj4I=\fof choices A = {ai}8\ni=1 to best complete the matrix. Permutation invariance is a unique property for\nRPM problems: (1) According to [17], the same set of rules is applied either row-wise or column-wise.\nTherefore, swapping the \ufb01rst two rows or columns should not affect how one solves the problem.\n(2) In any multi-choice task, changing the order of answer candidates should not affect how one\nsolves the problem either. These properties require us to use a permutation-invariant encoder and\nreformulate the problem from a typical classi\ufb01cation problem into a ranking problem. Formally, in a\nprobabilistic formulation, we seek to \ufb01nd a model such that\n\np(a(cid:63)|O) \u2265 p(a(cid:48)|O),\n\n(1)\nwhere the probability is invariant when rows or columns in O are swapped. This formulation also\ncalls for a model that produces a density estimation for each choice, regardless of its order in A.\nTo that end, we model the probability with a neural network equipped with a permutation-invariant\nencoder for each observation-candidate pair f (O \u222a a). However, we argue such a purely perceptive\nsystem is far from suf\ufb01cient without contrasting and perceptual inference.\n\n\u2200a(cid:48) \u2208 A, a(cid:48) (cid:54)= a(cid:63),\n\n3.1 Contrasting\n\nTo provide the reasoning system with a mechanism of contrasting, we propose to explicitly build two\nlevels of contrast: model-level contrast and objective-level contrast.\n\n3.1.1 Model-level Contrast\n\nAs the central notion of contrast is comparing cases [30, 32, 44, 45], we propose an explicit model-\nlevel contrasting mechanism in the following form,\n\n(cid:33)\n\n(cid:32)(cid:88)\n\na(cid:48)\u2208A\n\nContrast(FO\u222aa) = FO\u222aa \u2212 h\n\nFO\u222aa(cid:48)\n\n,\n\n(2)\n\nwhere F denotes features of a speci\ufb01c combination and h(\u00b7) summarizes the common features in all\ncandidate answers. In our experiments, h(\u00b7) is a composition of BatchNorm [67] and Conv.\nIntuitively, this explicit contrasting computation enables a reasoning system to tell distinguishing\nfeatures for each candidate in terms of \ufb01tting and following the rules hidden among all panels in\nthe incomplete matrix. The philosophy behind this design is to constrain the functional form of the\nmodel to capture both the commonality and the difference in each instance. It is expected that the\nvery inductive bias on comparing similarity and distinctness is baked into the entire reasoning system\nsuch that learning in the challenging task becomes easier.\nIn a generalized setting, each O \u222a a could be abstracted out as an object. Then the design becomes a\ngeneral contrast module, where each object is distinguished by comparing with the common features\nextracted from an object set.\nWe further note that the contrasting computation can be encapsulated into a single neural module\nand repeated: the addition and transformation are shared and the subtraction is performed on each\nindividual element. See Figure 1(c) for a sketch of the contrast module. After such operations,\npermutation invariance of a model will not be broken.\n\n3.1.2 Objective-level Contrast\n\nTo further enforce the contrast effects, we propose to use an NCE variant rather than the cross-entropy\nloss commonly used in previous works [12, 14]. While there are several ways to model the probability\nin Equation 1, we use a Gibbs distribution in this work:\n\np(a|O) =\n\n1\nZ\n\nexp(f (O \u222a a)),\n\n(3)\n\nwhere Z is the partition function, and our model f (\u00b7) corresponds to the negative potential function.\nNote that such a distribution has been widely adopted in image generation models [68\u201370].\nIn this case, we can take the log of both sides in Equation 1 and rearrange terms:\n\nlog p(a(cid:63)|O) \u2212 log p(a(cid:48)|O) = f (O \u222a a(cid:63)) \u2212 f (O \u222a a(cid:48)) \u2265 0,\n\n\u2200a(cid:48) \u2208 A, a(cid:48) (cid:54)= a(cid:63).\n\n(4)\n\n4\n\n\fThis formulation could potentially lead to a max margin loss. However, we notice in our preliminary\nexperiments that max margin is not suf\ufb01cient; we realize it is inferior to make the negative potential\nof the wrong choices only slightly lower. Instead, we would like to further push the difference to\nin\ufb01nity. To do that, we leverage the sigmoid function \u03c3(\u00b7) and train the model, such that:\n\nf (O \u222a a(cid:63)) \u2212 f (O \u222a a(cid:48)) \u2192 \u221e \u21d0\u21d2 \u03c3(f (O \u222a a(cid:63)) \u2212 f (O \u222a a(cid:48))) \u2192 1,\u2200a(cid:48) \u2208 A, a(cid:48) (cid:54)= a(cid:63).\n\n(5)\nHowever, we notice that the relative difference of negative potential is still problematic. We hypoth-\nesize this de\ufb01ciency is due to the lack of a baseline\u2014without such a regularization, the negative\npotential of wrong choices could still be very high, resulting in dif\ufb01culties in learning the negative\npotential of the correct answer. To this end, we modify Equation 5 into its suf\ufb01cient conditions:\n\nf (O \u222a a(cid:63)) \u2212 b(O \u222a a(cid:63)) \u2192 \u221e \u21d0\u21d2 \u03c3(f (O \u222a a(cid:63)) \u2212 b(O \u222a a(cid:63))) \u2192 1\nf (O \u222a a(cid:48)) \u2212 b(O \u222a a(cid:48)) \u2192 \u2212\u221e \u21d0\u21d2 \u03c3(f (O \u222a a(cid:48)) \u2212 b(O \u222a a(cid:48))) \u2192 0,\n\n(6)\n(7)\nwhere b(\u00b7) is a \ufb01xed baseline function and a(cid:48) \u2208 A, a(cid:48) (cid:54)= a(cid:63). For implementation, b(\u00b7) could be either\na randomly initialized network or a constant. Since the two settings do not produce signi\ufb01cantly\ndifferent results in our preliminary experiments, we set b(\u00b7) to be a constant to reduce computation.\nWe then optimize the network to maximize the following objective as done in [39]:\n\n(cid:96) = log(\u03c3(f (O \u222a a(cid:63)) \u2212 b(O \u222a a(cid:63)))) +\n\nlog(1 \u2212 \u03c3(f (O \u222a a(cid:48)) \u2212 b(O \u222a a(cid:48)))).\n(cid:54)=a(cid:63)\n\n(cid:48)\n\n(cid:48)\n\na\n\n\u2208A,a\n\n(8)\n\nConnection to NCE If we treat the baseline as the negative potential of a \ufb01xed noise model of the\nsame Gibbs form and ignore the difference between the partition functions, Equation 6 and Equation 7\nbecome the G function used in NCE [39]. But unlike NCE, we do not need to multiply the size ratio\nin the sigmoid function [41].\n\n3.2 Perceptual Inference\n\nAs indicated in Carpenter et al. [17], a mere perceptive model for RPM is arguably not enough.\nTherefore, we propose to incorporate a simple inference subsystem into the model: the inference\nbranch should be responsible for inferring the hidden rules in the problem. Speci\ufb01cally, we assume\nthere are at most N attributes in each problem, each of which is subject to the governance of one of\nM rules. Then hidden rules T in one problem instance can be decomposed into\n\n(cid:88)\n\nN(cid:89)\n\ni=1\n\np(T |O) =\n\np(ti|O),\n\n(9)\n\nwhere ti = 1 . . . M denotes the rule type on attribute ni. For the actual form of the probability of\nrules on each attribute, we propose to model it using a multinomial distribution. This assumption is\nconsistent with the way datasets are usually generated [12, 14, 61]: one rule is independently picked\nfrom the rule set for each attribute. In this way, each rule could also be regarded as a basis in a rule\ndictionary and jointly learned, as done in active basis [42] or word embedding [49, 71].\nIf we treat rules as hidden variables, the log probability in Equation 4 can be decomposed into\n\nlog p(a|O) = log\n\np(a|T ,O)p(T |O) = log E\n\nT \u223cp(T |O)[p(a|T ,O)].\n\n(10)\n\n(cid:88)\n\nT\n\nNote that writing the summation in the form of expectation affords sampling algorithms, which can\nbe done on each individual attribute due to the independence assumption.\nIn addition, if we model p(T |O) as an inference branch g(\u00b7) and sample only once from it, the model\ncan be modi\ufb01ed into f (O \u222a a, \u02c6T ) with \u02c6T sampled from g(O). Following the same derivation above,\nwe now optimize the new objective:\n\n(cid:96) = log(\u03c3(f (O \u222a a(cid:63), \u02c6T ) \u2212 b(O \u222a a(cid:63)))) +\n\nlog(1 \u2212 \u03c3(f (O \u222a a(cid:48), \u02c6T ) \u2212 b(O \u222a a(cid:48)))).\n(cid:54)=a(cid:63)\n\na\n\n(cid:48)\n\n(cid:48)\n\u2208A,a\n\n(11)\n\nTo sample from a multinomial, we could either use hard sampling like Gumbel-SoftMax [72, 73] or a\nsoft one by taking expectation. We do not observe signi\ufb01cant difference between the two settings.\n\n5\n\n(cid:88)\n\n\fThe expectation in Equation 10 is proposed primarily to make the computation of the exact log\nprobability controllable and tractable: while the full summation requires O(M N ) passes of the model,\na Monte Carlo approximation of it could be calculated in O(1) time. We also note that if p(T |O) is\nhighly peaked (e.g., ground truth), the Monte Carlo estimate could be accurate as well. Despite the\nfact that we only sample once from an inference branch to reduce computation, we \ufb01nd in practice\nthe Monte Carlo estimate works quite well.\n\n3.3 Architecture\n\nCombining contrasting, perceptual inference, and permutation invariance, we propose a new network\narchitecture to solve the challenging RPM problem, named Contrastive Perceptual Inference network\n(CoPINet). The perception branch is composed of a common feature encoder and shared interweaving\ncontrast modules and residual blocks [3]. The encoder \ufb01rst extracts image features independently for\neach panel and sum ones in the corresponding rows and columns before the \ufb01nal transformation into\na latent space. The inference branch consists of the same encoder and a (Gumbel-)SoftMax output\nlayer. The sampled results will be transformed and concatenated channel-wise into the summation in\nEquation 2. In our implementation, we prepend each residual block with a contrast module; such a\ncombination can be repeated while keeping the network permutation-invariant. The network \ufb01nally\nuses an MLP to produce a negative potential for each observation and candidate pair and is trained\nusing Equation 11; see Figure 1(b) for a graphical illustration of the entire CoPINet architecture.\n\n4 Experiments\n\n4.1 Experimental Setup\n\nWe verify the effectiveness of our models on two major RPM datasets: RAVEN [12] and PGM [14].\nAcross all experiments, we train models on the training set, tune hyper-parameters on the validation\nset, and report the \ufb01nal results on the test set. All of the models are implemented in PyTorch [74]\nand optimized using ADAM [75]. While a good performance of WReN [14] and ResNet+DRT [12]\nrelies on external supervision, such as rule speci\ufb01cations and structural annotations, the proposed\nmodel achieves better performance with only O, A, and a(cid:63). Models are trained on servers with\nfour Nvidia RTX Titans. For the WReN model, we use a public implementation that reproduces\nresults in [14]1. We implement our models in PyTorch [74] and optimize using ADAM [75]. During\ntraining, we perform early-stop based on validation loss. We use the same network architecture and\nhyper-parameters in both RAVEN and PGM experiments.\n\n4.2 Results on RAVEN\n\nThere are 70, 000 problems in the RAVEN dataset [12], equally distributed in 7 \ufb01gure con\ufb01gurations.\nIn each con\ufb01guration, the dataset is randomly split into 6 folds for training, 2 folds for validation,\nand 2 folds for testing. We compare our model with several simple baselines (LSTM [76], CNN [77],\nand vanilla ResNet [3]) and two strong baselines (WReN [14] and ResNet+DRT [12]). Model\nperformance is measured by accuracy.\n\nGeneral Performance on RAVEN In this experiment, we train the models on all 42, 000 training\nsamples and measure how they perform on the test set. The \ufb01rst part of Table 1 shows the testing\naccuracy of all models. We also retrieve the performance of humans and a solver with perfect\ninformation from [12] for comparison. As shown in the table, the proposed model CoPINet achieves\nthe best performance among all the models we test. For the relational model WReN proposed in [14],\nwe run the tests on a permutation-invariant version, i.e., one without positional tagging (NoTag),\nand tune the model also to minimize an auxiliary loss (Aux) [14]. While the auxiliary loss could\nboost the performance of WReN as we will show later in the ablation study, we do not observe\nsimilar effects on CoPINet. As indicated in the detailed comparisons in Table 1, WReN is biased\ntowards images of grid con\ufb01gurations and does poorly on ones demanding compositional reasoning,\ni.e., ones with independent components. We further note that compared to previously proposed\nmodels (WReN [14] and ResNet+DRT [12]), CoPINet does not require additional information such\nas structural annotations and meta targets and still shows human-level performance in this task. When\n\n1https://github.com/Fen9/WReN\n\n6\n\n\fcomparing the performance of CoPINet and human on speci\ufb01c \ufb01gure con\ufb01gurations, we notice that\nCoPINet is inferior in learning samples of grid-like compositionality but ef\ufb01cient in distinguishing\nimages consisting of multiple components, implying the ef\ufb01ciency of the contrasting mechanism.\n\nAblation Study One problem of particular interest in building CoPINet is how each component\ncontributes to performance improvement. To answer this question, we measure model accuracy by\ngradually removing each construct in CoPINet, i.e., the perceptual inference branch, the contrast\nloss, and the contrast module. In the second part of Table 1, we show the results of ablation on\nCoPINet. Both the full model (CoPINet) and the one without the perceptual inference branch\n(CoPINet-Contrast-CL) could achieve human-level performance, with the latter slightly inferior to the\nformer. If we further replace the contrast loss with the cross-entropy loss (CoPINet-Contrast-XE), we\nobserve a noticeable performance decrease of around 4%, verifying the effectiveness of the contrast\nloss. A catastrophic performance downgrade of 66% is observed if we remove the contrast module,\nleaving only the network backbone (CoPINet-Backbone-XE). This drastic performance gap shows\nthat the functional constraint on modeling an explicit contrasting mechanism is arguably a crucial\nfactor in machines\u2019 reasoning ability as well as in humans\u2019. The ablation study shows that all the three\nproposed constructs, especially the contrast module, are critical to the performance of CoPINet. We\nalso study how the requirement of permutation invariance and auxiliary training affect the previously\nproposed WReN. As shown in Table 1, sacri\ufb01cing the permutation invariance (Tag) provides the\nmodel a huge upgrade during auxiliary training (Aux), compared to the one without tagging (NoTag)\nand auxiliary loss (NoAux). This effect becomes even more signi\ufb01cant on the PGM dataset, as we\nwill show in Section 4.3.\n\nDataset Size and Performance Even though CoPINet surpasses human performance on RAVEN,\nthis competition is inherently unfair, as the human subjects in this study never experience such an\nintensive training session as our model does. To make the comparison fairer and also as a step towards\na model capable of human learning ef\ufb01ciency, we further measure how the model performance\nchanges as the training set size shrinks. To this end, we train our CoPINet on subsets of the full\nRAVEN training set and test it on the full test set. As shown on Table 2 and Figure 2, the model\nperformance varies roughly log-linearly with the training set size. One surprising observation is:\nwith only half of the amount of the data, we could already achieve human-level performance. On a\ntraining set 16\u00d7 smaller, CoPINet outperforms all previous models. And on a subset 64\u00d7 smaller,\nCoPINet already outshines WReN.\n\n4.3 Results on PGM\n\nWe use the neutral regime of the PGM dataset for model evaluation due to its diversity and richness in\nrelationships, objects, and attributes. This split of the dataset has in total 1.42 million samples, with\n1.2 million for training, 2, 000 for validation, and 200, 000 for testing. We train the models on the\ntraining set, tune the hyperparameters on the validation set, and evaluate the performance on the test\n\nTable 1: Testing accuracy of models on RAVEN. Acc denotes the mean accuracy of each model. Same as in [12],\nL-R denotes the Left-Right con\ufb01guration, U-D Up-Down, O-IC Out-InCenter, and O-IG Out-InGrid.\n\nMethod\nLSTM\nWReN-NoTag-Aux\nCNN\nResNet\nResNet+DRT\nCoPINet\nWReN-NoTag-NoAux\nWReN-Tag-NoAux\nWReN-Tag-Aux\nCoPINet-Backbone-XE\nCoPINet-Contrast-XE\nCoPINet-Contrast-CL\nHuman\nSolver\n\nAcc\n\nCenter\n\n2x2Grid\n\n3x3Grid\n\nL-R\n\nU-D\n\nO-IC\n\nO-IG\n\n13.07% 13.19% 14.13% 13.69% 12.84% 12.35% 12.15% 12.99%\n17.62% 17.66% 29.02% 34.67%\n12.30% 13.94%\n36.97% 33.58% 30.30% 33.53% 39.43% 41.26% 43.20% 37.54%\n53.43% 52.82% 41.86% 44.29% 58.77% 60.16% 63.19% 53.12%\n59.56% 58.08% 46.53% 50.40% 65.82% 67.11% 69.09% 60.11%\n91.42% 95.05% 77.45% 78.85% 99.10% 99.65% 98.50% 91.35%\n\n7.89%\n\n7.69%\n\n15.07% 12.30% 28.62% 29.22%\n13.10%\n17.94% 15.38% 29.81% 32.94% 11.06% 10.96% 11.06% 14.54%\n33.97% 58.38% 38.89% 37.70% 21.58% 19.74% 38.84% 22.57%\n20.75% 24.00% 23.25% 23.05% 15.00% 13.90% 21.25% 24.80%\n86.16% 87.25% 71.05% 74.45% 97.25% 97.05% 93.20% 82.90%\n90.04% 94.30% 74.00% 76.85% 99.05% 99.35% 98.00% 88.70%\n\n7.20%\n\n8.33%\n\n6.55%\n\n84.41% 95.45% 81.82% 79.55% 86.36% 81.81% 86.36% 81.81%\n100%\n100%\n\n100%\n\n100%\n\n100%\n\n100%\n\n100%\n\n100%\n\n7\n\n\fFigure 2: CoPINet on RAVEN\nand PGM as the training set size\nshrinks.\n\nTable 2: Model performance under\ndifferent training set sizes on RAVEN\ndataset. The full training set has\n42, 000 samples.\n\nTable 3: Model performance under\ndifferent training set sizes on PGM\ndataset. The full training set has 1.2\nmillion samples.\n\nTraining set size\n\nAcc\n\nTraining set size\n\nAcc\n\n658\n1, 316\n2, 625\n5, 250\n10, 500\n21, 000\n\n44.48%\n57.69%\n65.55%\n74.53%\n80.92%\n86.43%\n\n293\n1, 172\n4, 688\n18, 750\n75, 000\n300, 000\n\n14.73%\n15.48%\n18.39%\n22.07%\n32.39%\n43.89%\n\nTable 4: Testing accuracy of models on PGM. Acc denotes the mean accuracy of each model.\n\nMethod\nAcc\n\nCNN\n\nLSTM ResNet Wild-ResNet WReN-NoTag-Aux CoPINet\n56.37%\n\n49.10%\n\n48.00%\n\n33.00% 35.80% 42.00%\n\nset. We compare our models with baselines set up in [14], i.e., LSTM, CNN, ResNet, Wild-ResNet,\nand WReN. As ResNet+DRT proposed in [12] requires structural annotations not available in PGM,\nwe are unable to measure its performance. Again, all performance is measured by accuracy. Due to\nthe lack of further strati\ufb01cation on this training regime, we only report the \ufb01nal mean accuracy.\n\nGeneral Performance on PGM In this experiment, we train the models on all 1.2 million training\nsamples and report performance on the entire test set. As shown in Table 4, CoPINet achieves the best\nperformance among all permutation-invariant models, setting a new state-of-the-art on this dataset.\nSimilar to the setting in RAVEN, we make the previously proposed WReN permutation-invariant by\nremoving the positional tagging (NoTag) and train it with both cross-entropy loss and auxiliary loss\n(Aux) [14]. The auxiliary loss could boost the performance of WReN. However, in coherence with\nthe study on RAVEN and a previous work [12], we notice that the auxiliary loss does not help our\nCoPINet. It is worth noting that while WReN demands additional training supervision from meta\ntargets to reach the performance, CoPINet only requires basic annotations of ground truth indices a(cid:63)\nand achieves better results.\n\nAblation Study We perform ablation studies on both WReN and CoPINet to see how the require-\nment of permutation invariance affects WReN and how each module in CoPINet contributes to its\nsuperior performance. The notations are the same as those used in the ablation study for RAVEN.\nAs shown in the \ufb01rst part of Table 5, adding a proper auxiliary loss does provide WReN a 10%\nperformance boost. However, additional supervision is required. Making the model permutation-\nsensitive gives the model a signi\ufb01cant bene\ufb01t by up to a 28% accuracy increase; however, it also\nindicates that WReN learns to shortcut the solutions by coding the positional association, instead\nof truly understanding the differences among distinctive choices and their potential effects on the\ncompatibility of the entire matrix. The second part of Table 5 demonstrates how each construct\ncontributes to the performance improvement of CoPINet on PGM. Despite the smaller enhancement\nof the contrast loss compared to that in RAVEN, the upgrade from the contrast module for PGM is still\nsigni\ufb01cant, and the perceptual inference branch keeps raising the \ufb01nal performance. In accordance\nwith the ablation study on the RAVEN dataset, we show that all the proposed components contribute\nto the \ufb01nal performance increase.\n\nTable 5: Ablation study on PGM.\n\nWReN-NoTag-Aux\n\nWReN-Tag-NoAux WReN-Tag-Aux\n\n49.10%\n\n62.45%\n\n77.94%\n\nCoPINet\n\n56.37%\n\nMethod WReN-NoTag-NoAux\nAcc\n\n39.25%\n\nMethod CoPINet-Backbone-XE CoPINet-Contrast-XE CoPINet-Contrast-CL\nAcc\n\n54.19%\n\n42.10%\n\n51.04%\n\n8\n\n102104106Trainingsetsize20406080AccRAVENPGM\fDataset Size and Performance Motivated by the idea of fairer comparison and low-shot reasoning,\nwe also measure how the performance of the proposed CoPINet changes as the training set size of\nPGM varies. Speci\ufb01cally, we train CoPINet on subsets of the PGM training set and test it on the\nentire test set. As shown in Table 3 and Figure 2, CoPINet performance on PGM varies roughly\nlog-exponentially with respect to the training set size. We further note that when trained on a 16\u00d7\nsmaller dataset, CoPINet already achieves results similar to CNN and LSTM.\n\n5 Conclusion and Discussion\n\nIn this work, we aim to improve machines\u2019 reasoning ability in \u201cthinking in pictures\u201d by jointly\nlearning perception and inference via contrasting. Speci\ufb01cally, we introduce the contrast module,\nthe contrast loss, and the joint system of perceptual inference. We also require our model to be\npermutation-invariant. In a typical and challenging task of this kind, Raven\u2019s Progressive Matri-\nces (RPM), we demonstrate that our proposed model\u2014Contrastive Perceptual Inference network\n(CoPINet)\u2014achieves the new state-of-the-art for permutation-invariant models on two major RPM\ndatasets. Further ablation studies show that all the three proposed components are effective towards\nimproving the \ufb01nal results, especially the contrast module. It also shows that the permutation invari-\nance forces the model to understand the effects of different choices on the compatibility of an entire\nRPM matrix, rather than remembering the positional association and shortcutting the solutions.\nWhile it is encouraging to see the performance improvement of the proposed ideas on two big datasets,\nit is the last part of the experiments, i.e., dataset size and performance, that really intrigues us. With\nin\ufb01nitely large datasets that cover the entirety of an arbitrarily complex problem domain, it is arguably\npossible that a simple over-parameterized model could solve it. However, in reality, there is barely\nany chance that one would observe all the domain, yet humans still learn quite ef\ufb01ciently how the\nhidden rules work. We believe this is the core where the real intelligence lies: learning from only\na few samples and generalizing to the extreme. Even though CoPINet already demonstrates better\nlearning ef\ufb01ciency, it would be ideal to have models capable of few-shot learning in the task of RPM.\nWithout massive datasets, it would be a real challenge, and we hope the paper could call for future\nresearch into it.\nPerformance, however, is de\ufb01nitely not the end goal in the line of research on relational and analogical\nvisual reasoning: other dimensions for measurements include generalization, generability, and\ntransferability. Is it possible for a model to be trained on a single con\ufb01guration and generalize to\nother settings? Can we generate the \ufb01nal answer based on the given context panels, in a similar way\nto the top-down and bottom-up method jointly applied by humans for reasoning? Can we transfer\nthe relational and geometric knowledge required in the reasoning task from other tasks? Questions\nlike these are far from being answered. While Zhang et al. [12] show in the experiments that neural\nmodels do possess a certain degree of generalizability, the testing accuracy is far from satisfactory. In\nthe meantime, there are a plethora of discriminative approaches towards solving reasoning problems\nin question answering, but generative methods and combined methods are lacking. The relational and\nanalogical reasoning was initially introduced as a way to measure a human\u2019s intelligence, without\ntraining humans on the task. However, current settings uniformly reformulate it as a learning problem\nrather than a transfer problem, contradictory to why the task was started. Up to now, there has been\nbarely any work that measures how knowledge on another task could be transferred to this one. We\nbelieve that signi\ufb01cant advances in these dimensions would possibly enable Arti\ufb01cial Intelligence (AI)\nmodels to go beyond data \ufb01tting and acquire symbolized knowledge.\nWhile modern computer vision techniques to solve Raven\u2019s Progressive Matrices (RPM) are based\non neural networks, a promising ingredient is nowhere to be found: Gestalt psychology. Traces of the\nperceptual grouping and \ufb01gure-ground organization are gradually faded out in the most recent wave\nof deep learning. However, the principles of grouping, both classical (e.g., proximity, closure, and\nsimilarity) and new (e.g., synchrony, element, and uniform connectedness) play an essential role in\nRPM, as humans arguably solve these problems by \ufb01rst \ufb01guring out groups and then applying the\nrules. We anticipate that modern deep learning methods integrated with the tradition of conceptual and\ntheoretical foundations of the Gestalt approach would further improve models on abstract reasoning\ntasks like RPM.\nAcknowledgments: This work reported herein is supported by MURI ONR N00014-16-1-2007,\nDARPA XAI N66001-17-2-4029, ONR N00014-19-1-2153, NSF BSC-1827374, and an NVIDIA\nGPU donation grant.\n\n9\n\n\fReferences\n[1] Temple Grandin. Thinking in pictures: And other reports from my life with autism. Vintage,\n\n2006.\n\n[2] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classi\ufb01cation with deep\nconvolutional neural networks. In Proceedings of Advances in Neural Information Processing\nSystems (NIPS), 2012.\n\n[3] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for im-\nage recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern\nRecognition (CVPR), 2016.\n\n[4] Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Uni\ufb01ed,\nreal-time object detection. In Proceedings of the IEEE Conference on Computer Vision and\nPattern Recognition (CVPR), 2016.\n\n[5] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object\ndetection with region proposal networks. In Proceedings of Advances in Neural Information\nProcessing Systems (NIPS), 2015.\n\n[6] Stanislaw Antol, Aishwarya Agrawal,\n\nC Lawrence Zitnick, and Devi Parikh. Vqa: Visual question answering.\nof International Conference on Computer Vision (ICCV), 2015.\n\nJiasen Lu, Margaret Mitchell, Dhruv Batra,\nIn Proceedings\n\n[7] Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Li Fei-Fei, C Lawrence Zitnick,\nand Ross Girshick. Clevr: A diagnostic dataset for compositional language and elementary\nvisual reasoning. In Proceedings of the IEEE Conference on Computer Vision and Pattern\nRecognition (CVPR), 2017.\n\n[8] Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Judy Hoffman, Li Fei-Fei,\nC Lawrence Zitnick, and Ross B Girshick. Inferring and executing programs for visual reasoning.\nIn Proceedings of International Conference on Computer Vision (ICCV), 2017.\n\n[9] Ronghang Hu, Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Kate Saenko. Learning\nto reason: End-to-end module networks for visual question answering. In Proceedings of\nInternational Conference on Computer Vision (ICCV), 2017.\n\n[10] Rudolf Arnheim. Visual thinking. Univ of California Press, 1969.\n[11] Francis Galton. Inquiries into human faculty and its development. Macmillan, 1883.\n[12] Chi Zhang, Feng Gao, Baoxiong Jia, Yixin Zhu, and Song-Chun Zhu. Raven: A dataset for\nrelational and analogical visual reasoning. In Proceedings of the IEEE Conference on Computer\nVision and Pattern Recognition (CVPR), 2019.\n\n[13] Felix Hill, Adam Santoro, David GT Barrett, Ari S Morcos, and Timothy Lillicrap. Learning to\n\nmake analogies by contrasting abstract relational structure. arXiv:1902.00120, 2019.\n\n[14] David Barrett, Felix Hill, Adam Santoro, Ari Morcos, and Timothy Lillicrap. Measuring abstract\nreasoning in neural networks. In Proceedings of International Conference on Machine Learning\n(ICML), 2018.\n\n[15] James C Raven. Mental tests used in genetic studies: The performance of related individuals on\ntests mainly educative and mainly reproductive. Master\u2019s thesis, University of London, 1936.\n[16] John C Raven and John Hugh Court. Raven\u2019s progressive matrices and vocabulary scales.\n\nOxford pyschologists Press, 1998.\n\n[17] Patricia A Carpenter, Marcel A Just, and Peter Shell. What one intelligence test measures:\na theoretical account of the processing in the raven progressive matrices test. Psychological\nreview, 97(3):404, 1990.\n\n[18] R E Snow, Patrick Kyllonen, and B Marshalek. The topography of ability and learning\n\ncorrelations. Advances in the psychology of human intelligence, pages 47\u2013103, 1984.\n\n[19] Charles Spearman. The abilities of man. Macmillan, 1927.\n\n10\n\n\f[20] Charles Spearman. The nature of \"intelligence\" and the principles of cognition. Macmillan,\n\n1923.\n\n[21] Douglas R Hofstadter. Fluid concepts and creative analogies: Computer models of the funda-\n\nmental mechanisms of thought. Basic books, 1995.\n\n[22] Susanne M Jaeggi, Martin Buschkuehl, John Jonides, and Walter J Perrig. Improving \ufb02uid\nintelligence with training on working memory. Proceedings of the National Academy of Sciences,\n105(19):6829\u20136833, 2008.\n\n[23] James J Gibson and Eleanor J Gibson. Perceptual learning: Differentiation or enrichment?\n\nPsychological review, 62(1):32, 1955.\n\n[24] James J Gibson. The ecological approach to visual perception: classic edition. Psychology\n\nPress, 2014.\n\n[25] Richard Catrambone and Keith J Holyoak. Overcoming contextual limitations on problem-\nsolving transfer. Journal of Experimental Psychology: Learning, Memory, and Cognition, 15\n(6):1147, 1989.\n\n[26] Dedre Gentner and Virginia Gunn. Structural alignment facilitates the noticing of differences.\n\nMemory & Cognition, 29(4):565\u2013577, 2001.\n\n[27] Rubi Hammer, Gil Diesendruck, Daphna Weinshall, and Shaul Hochstein. The development of\n\ncategory learning strategies: What makes the difference? Cognition, 112(1):105\u2013119, 2009.\n\n[28] Mary L Gick and Katherine Paterson. Do contrasting examples facilitate schema acquisition\nand analogical transfer? Canadian Journal of Psychology/Revue canadienne de psychologie, 46\n(4):539, 1992.\n\n[29] Etsuko Haryu, Mutsumi Imai, and Hiroyuki Okada. Object similarity bootstraps young children\n\nto action-based verb extension. Child Development, 82(2):674\u2013686, 2011.\n\n[30] Linsey Smith and Dedre Gentner. The role of difference-detection in learning contrastive\ncategories. In Proceedings of the Annual Meeting of the Cognitive Science Society (CogSci),\n2014.\n\n[31] Dedre Gentner. Structure-mapping: A theoretical framework for analogy. Cognitive science, 7\n\n(2):155\u2013170, 1983.\n\n[32] Daniel L Schwartz, Catherine C Chase, Marily A Oppezzo, and Doris B Chin. Practicing versus\ninventing with contrasting cases: The effects of telling \ufb01rst on learning and transfer. Journal of\nEducational Psychology, 103(4):759, 2011.\n\n[33] Gordon H Bower. A contrast effect in differential conditioning. Journal of Experimental\n\nPsychology, 62(2):196, 1961.\n\n[34] Donald R Meyer. The effects of differential rewards on discrimination reversal learning by\n\nmonkeys. Journal of Experimental Psychology, 41(4):268, 1951.\n\n[35] Allan M Schrier and Harry F Harlow. Effect of amount of incentive on discrimination learning\n\nby monkeys. Journal of comparative and physiological psychology, 49(2):117, 1956.\n\n[36] Robert M Shapley and Jonathan D Victor. The effect of contrast on the transfer properties of cat\n\nretinal ganglion cells. The Journal of physiology, 285(1):275\u2013298, 1978.\n\n[37] Reed Lawson. Brightness discrimination performance and secondary reward strength as a\nfunction of primary reward amount. Journal of Comparative and Physiological Psychology, 50\n(1):35, 1957.\n\n[38] Abram Amsel. Frustrative nonreward in partial reinforcement and discrimination learning:\n\nSome recent history and a theoretical extension. Psychological review, 69(4):306, 1962.\n\n[39] Michael Gutmann and Aapo Hyv\u00e4rinen. Noise-contrastive estimation: A new estimation\nprinciple for unnormalized statistical models. In Proceedings of the International Conference\non Arti\ufb01cial Intelligence and Statistics (AISTATS), 2010.\n\n11\n\n\f[40] Noah A Smith and Jason Eisner. Contrastive estimation: Training log-linear models on unlabeled\ndata. In Proceedings of the Annual Meeting of the Association for Computational Linguistics\n(ACL), 2005.\n\n[41] Bo Dai and Dahua Lin. Contrastive learning for image captioning. In Proceedings of Advances\n\nin Neural Information Processing Systems (NIPS), 2017.\n\n[42] Ying Nian Wu, Zhangzhang Si, Haifeng Gong, and Song-Chun Zhu. Learning active basis\nmodel for object detection and recognition. International Journal of Computer Vision (IJCV),\n90(2):198\u2013235, 2010.\n\n[43] Dedre Gentner and Arthur B Markman. Structural alignment in comparison: No difference\n\nwithout similarity. Psychological science, 5(3):152\u2013158, 1994.\n\n[44] Catherine C Chase, Jonathan T Shemwell, and Daniel L Schwartz. Explaining across con-\ntrasting cases for deep understanding in science: An example using interactive simulations. In\nProceedings of the 9th International Conference of the Learning Sciences, 2010.\n\n[45] Daniel L Schwartz and Taylor Martin. Inventing to prepare for future learning: The hidden\nef\ufb01ciency of encouraging original student production in statistics instruction. Cognition and\nInstruction, 22(2):129\u2013184, 2004.\n\n[46] Sumit Chopra, Raia Hadsell, Yann LeCun, et al. Learning a similarity metric discriminatively,\nwith application to face veri\ufb01cation. In Proceedings of the IEEE Conference on Computer\nVision and Pattern Recognition (CVPR), 2005.\n\n[47] Kilian Q Weinberger and Lawrence K Saul. Distance metric learning for large margin nearest\n\nneighbor classi\ufb01cation. Journal of Machine Learning Research, 10(Feb):207\u2013244, 2009.\n\n[48] Xiaolong Wang and Abhinav Gupta. Unsupervised learning of visual representations using\n\nvideos. In Proceedings of International Conference on Computer Vision (ICCV), 2015.\n\n[49] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed represen-\ntations of words and phrases and their compositionality. In Proceedings of Advances in Neural\nInformation Processing Systems (NIPS), 2013.\n\n[50] Ryan Kiros, Yukun Zhu, Ruslan R Salakhutdinov, Richard Zemel, Raquel Urtasun, Antonio\nIn Proceedings of Advances in Neural\n\nTorralba, and Sanja Fidler. Skip-thought vectors.\nInformation Processing Systems (NIPS), 2015.\n\n[51] David J Chalmers, Robert M French, and Douglas R Hofstadter. High-level perception, represen-\ntation, and analogy: A critique of arti\ufb01cial intelligence methodology. Journal of Experimental\n& Theoretical Arti\ufb01cial Intelligence, 4(3):185\u2013211, 1992.\n\n[52] Melanie Mitchell. Analogy-making as perception: A computer model. MIT Press, 1993.\n[53] Andrew Lovett and Kenneth Forbus. Modeling visual problem solving as analogical reasoning.\n\nPsychological Review, 124(1):60, 2017.\n\n[54] Andrew Lovett, Kenneth Forbus, and Jeffrey Usher. A structure-mapping model of raven\u2019s\nprogressive matrices. In Proceedings of the Annual Meeting of the Cognitive Science Society\n(CogSci), 2010.\n\n[55] Andrew Lovett, Emmett Tomai, Kenneth Forbus, and Jeffrey Usher. Solving geometric analogy\nproblems through two-stage analogical mapping. Cognitive science, 33(7):1192\u20131231, 2009.\n[56] Daniel R Little, Stephan Lewandowsky, and Thomas L Grif\ufb01ths. A bayesian model of rule\ninduction in raven\u2019s progressive matrices. In Proceedings of the Annual Meeting of the Cognitive\nScience Society (CogSci), 2012.\n\n[57] Keith McGreggor and Ashok K Goel. Con\ufb01dent reasoning on raven\u2019s progressive matrices tests.\n\nIn Proceedings of AAAI Conference on Arti\ufb01cial Intelligence (AAAI), 2014.\n\n[58] Keith McGreggor, Maithilee Kunda, and Ashok Goel. Fractals and ravens. Arti\ufb01cial Intelligence,\n\n215:1\u201323, 2014.\n\n12\n\n\f[59] Can Serif Mekik, Ron Sun, and David Yun Dai. Similarity-based reasoning, raven\u2019s matri-\nces, and general intelligence. In Proceedings of International Joint Conference on Arti\ufb01cial\nIntelligence (IJCAI), 2018.\n\n[60] Snejana Shegheva and Ashok Goel. The structural af\ufb01nity method for solving the raven\u2019s\nprogressive matrices test for intelligence. In Proceedings of AAAI Conference on Arti\ufb01cial\nIntelligence (AAAI), 2018.\n\n[61] Ke Wang and Zhendong Su. Automatic generation of raven\u2019s progressive matrices. In Proceed-\n\nings of International Joint Conference on Arti\ufb01cial Intelligence (IJCAI), 2015.\n\n[62] Adam Santoro, David Raposo, David G Barrett, Mateusz Malinowski, Razvan Pascanu, Peter\nBattaglia, and Tim Lillicrap. A simple neural network module for relational reasoning. In\nProceedings of Advances in Neural Information Processing Systems (NIPS), 2017.\n\n[63] Xander Steenbrugge, Sam Leroux, Tim Verbelen, and Bart Dhoedt. Improving generaliza-\ntion for abstract reasoning tasks using disentangled feature representations. arXiv preprint\narXiv:1811.04784, 2018.\n\n[64] Song-Chun Zhu, David Mumford, et al. A stochastic grammar of images. Foundations and\n\nTrends R(cid:13) in Computer Graphics and Vision, 2(4):259\u2013362, 2007.\n[65] Seyoung Park and Song-Chun Zhu. Attributed grammars for joint estimation of human attributes,\npart and pose. In Proceedings of International Conference on Computer Vision (ICCV), 2015.\n[66] Tian-Fu Wu, Gui-Song Xia, and Song-Chun Zhu. Compositional boosting for computing\nhierarchical image structures. In Proceedings of the IEEE Conference on Computer Vision and\nPattern Recognition (CVPR), 2007.\n\n[67] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training\nby reducing internal covariate shift. In Proceedings of International Conference on Machine\nLearning (ICML), 2015.\n\n[68] Song Chun Zhu, Yingnian Wu, and David Mumford. Filters, random \ufb01elds and maximum\nInternational Journal of\n\nentropy (frame): Towards a uni\ufb01ed theory for texture modeling.\nComputer Vision, 27(2):107\u2013126, 1998.\n\n[69] Ying Nian Wu, Jianwen Xie, Yang Lu, and Song-Chun Zhu. Sparse and deep generalizations of\n\nthe frame model. Annals of Mathematical Sciences and Applications, 3(1):211\u2013254, 2018.\n\n[70] Jianwen Xie, Yang Lu, Song-Chun Zhu, and Yingnian Wu. A theory of generative convnet. In\n\nProceedings of International Conference on Machine Learning (ICML), 2016.\n\n[71] Jeffrey Pennington, Richard Socher, and Christopher Manning. Glove: Global vectors for word\nrepresentation. In Proceedings of the conference on Empirical Methods in Natural Language\nProcessing (EMNLP), 2014.\n\n[72] Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax.\n\narXiv:1611.01144, 2016.\n\n[73] Chris J Maddison, Andriy Mnih, and Yee Whye Teh. The concrete distribution: A continuous\n\nrelaxation of discrete random variables. arXiv:1611.00712, 2016.\n\n[74] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito,\nZeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in\npytorch. In NIPS-W, 2017.\n\n[75] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. International\n\nConference on Learning Representations (ICLR), 2014.\n\n[76] Sepp Hochreiter and J\u00fcrgen Schmidhuber. Long short-term memory. Neural computation,\n\n1997.\n\n[77] Dokhyam Hoshen and Michael Werman. Iq of neural networks. arXiv:1710.01692, 2017.\n\n13\n\n\f", "award": [], "sourceid": 634, "authors": [{"given_name": "Chi", "family_name": "Zhang", "institution": "University of California, Los Angeles"}, {"given_name": "Baoxiong", "family_name": "Jia", "institution": "UCLA"}, {"given_name": "Feng", "family_name": "Gao", "institution": "UCLA"}, {"given_name": "Yixin", "family_name": "Zhu", "institution": "University of California, Los Angeles"}, {"given_name": "HongJing", "family_name": "Lu", "institution": "UCLA"}, {"given_name": "Song-Chun", "family_name": "Zhu", "institution": "UCLA"}]}