The paper presents an extension to layer norm and provides theoretical analysis to the normalization scheme. There was concern about the direct applicability of the proposed AdaNorm, but general consensus is that the paper adds an interesting contribution to the understanding of LayerNorm related methods.