%PDF-1.3 1 0 obj << /Kids [ 4 0 R 5 0 R 6 0 R 7 0 R 8 0 R 9 0 R 10 0 R 11 0 R 12 0 R ] /Type /Pages /Count 9 >> endobj 2 0 obj << /Subject (Neural Information Processing Systems http\072\057\057nips\056cc\057) /Publisher (Curran Associates\054 Inc\056) /Language (en\055US) /Created (2015) /EventType (Poster) /Description-Abstract (We study optimization algorithms based on variance reduction for stochastic gradientdescent \050SGD\051\056 Remarkable recent progress has been made in this directionthrough development of algorithms like SAG\054 SVRG\054 SAGA\056 These algorithmshave been shown to outperform SGD\054 both theoretically and empirically\056 However\054asynchronous versions of these algorithms\204a crucial requirement for modernlarge\055scale applications\204have not been studied\056 We bridge this gap by presentinga unifying framework that captures many variance reduction techniques\056Subsequently\054 we propose an asynchronous algorithm grounded in our framework\054with fast convergence rates\056 An important consequence of our general approachis that it yields asynchronous versions of variance reduction algorithms such asSVRG\054 SAGA as a byproduct\056 Our method achieves near linear speedup in sparsesettings common to machine learning\056 We demonstrate the empirical performanceof our method through a concrete realization of asynchronous SVRG\056) /Producer (PyPDF2) /Title (On Variance Reduction in Stochastic Gradient Descent and its Asynchronous Variants) /Date (2015) /ModDate (D\07220161116134844\05508\04700\047) /Published (2015) /Type (Conference Proceedings) /firstpage (2647) /Book (Advances in Neural Information Processing Systems 28) /Description (Paper accepted and presented at the Neural Information Processing Systems Conference \050http\072\057\057nips\056cc\057\051) /Editors (C\056 Cortes and N\056D\056 Lawrence and D\056D\056 Lee and M\056 Sugiyama and R\056 Garnett) /Author (Sashank J\056 Reddi\054 Ahmed Hefny\054 Suvrit Sra\054 Barnabas Poczos\054 Alexander J\056 Smola) /lastpage (2655) >> endobj 3 0 obj << /Type /Catalog /Pages 1 0 R >> endobj 4 0 obj << /Parent 1 0 R /Contents 13 0 R /Type /Page /Resources 14 0 R /MediaBox [ 0 0 612 792 ] >> endobj 5 0 obj << /Parent 1 0 R /Contents 118 0 R /Type /Page /Resources 119 0 R /MediaBox [ 0 0 612 792 ] >> endobj 6 0 obj << /Parent 1 0 R /Contents 147 0 R /Type /Page /Resources 148 0 R /MediaBox [ 0 0 612 792 ] >> endobj 7 0 obj << /Parent 1 0 R /Contents 168 0 R /Type /Page /Resources 169 0 R /MediaBox [ 0 0 612 792 ] >> endobj 8 0 obj << /Parent 1 0 R /Contents 174 0 R /Type /Page /Resources 175 0 R /MediaBox [ 0 0 612 792 ] >> endobj 9 0 obj << /Parent 1 0 R /Contents 183 0 R /Type /Page /Resources 184 0 R /MediaBox [ 0 0 612 792 ] >> endobj 10 0 obj << /Parent 1 0 R /Contents 198 0 R /Type /Page /Resources 199 0 R /MediaBox [ 0 0 612 792 ] >> endobj 11 0 obj << /Parent 1 0 R /Contents 216 0 R /Type /Page /Resources 217 0 R /MediaBox [ 0 0 612 792 ] >> endobj 12 0 obj << /Parent 1 0 R /Contents 230 0 R /Type /Page /Resources 231 0 R /MediaBox [ 0 0 612 792 ] >> endobj 13 0 obj << /Length 3906 /Filter /FlateDecode >> stream xZ[o~`( f83M6&@fl<m1+ZK3ME-( ?"9_IM]n~)We,+_(tg"_]g~ r?ypoݟ]ʽQzVn&]/o|.۷ʺ_V'ǻ6ormiMY컕{͓Tҏ$x 3_Ef<|[]k_Y"uz_g ~}l |+w]ˊ%;,132 |fn)_e{*oWϷ!M9PE-lJ?TBCI"rwn7=vX^X4m=7vxJPn?7 k߿ϫu믫X7B""4n*_TO%j[nZ<9ye1䢤P˅D.6eAp,?IH _ʼnX+|Kυl>*V~_3BYiگ|T /+owy-{scN}owY B]r-XAIkќZ3EUvl,˲\DTn˗99;yߝ1)b?F$T؝cd Iғ1SOGQTR1t ʅXWճC_ @!"ô_+Vƫ]wkU;+*cn#ŏ3^!Ɖˉ`㋖aX!#ӕy$q#q0qk&_#~<i`%FZ:<1A@=teU۲仇Ak[sЛաAjElaP62ulvRg&:z0(EbSTަ/7]ֻ~vE?-yHxxMa~hꇦhs(s~]bTݶWoJ 38 m![(hM}|Z hq=.x5g <9Sm]rF@=ׯV0{ nގ~g{JoEۃ\*BJә%Enk}doW7@0ۢn TnGn*P6K͡Ɓl_lxA8 选\ĶVR:
> OlD,-~-m)Gr";dדzѢC-X2RYic ߬C#gm
jwh+ɃחաnA,tkNZϏžh.
EAoCJ!"vۜiR:+>na~(WXi
0D03epBm߬=w_ȧB?Ne3B&3p~>Bo:2!j:Cp=[%cn[SӫQԿӲ/)ꂄ]ɿT{(P1逄dY2¶RCh_L/5AD뺪X~q dD{!EQh:]'#r@>iZ|H:_d"0W4,>jjC?9