NeurIPS 2019
Sun Dec 8th through Sat the 14th, 2019 at Vancouver Convention Center
Paper ID:2446
Title:Understanding and Improving Layer Normalization

The paper presents an extension to layer norm and provides theoretical analysis to the normalization scheme. There was concern about the direct applicability of the proposed AdaNorm, but general consensus is that the paper adds an interesting contribution to the understanding of LayerNorm related methods.