{"title": "Data Poisoning Attacks on Factorization-Based Collaborative Filtering", "book": "Advances in Neural Information Processing Systems", "page_first": 1885, "page_last": 1893, "abstract": "Recommendation and collaborative filtering systems are important in modern information and e-commerce applications.  As these systems are becoming increasingly popular in industry, their outputs could affect business decision making, introducing incentives for an adversarial party to compromise the availability or integrity of such systems. We introduce a data poisoning attack on collaborative filtering systems.  We demonstrate how a powerful attacker with full knowledge of the learner can generate malicious data so as to maximize his/her malicious objectives, while at the same time mimicking normal user behaviors to avoid being detected. While the complete knowledge assumption seems extreme, it enables a robust assessment of the vulnerability of collaborative filtering schemes to highly motivated attacks. We present efficient solutions for two popular factorization-based collaborative filtering algorithms: the alternative minimization formulation and the nuclear norm minimization method. Finally, we test the effectiveness of our proposed algorithms on real-world data and discuss potential defensive strategies.", "full_text": "DataPoisoningAttacksonFactorization-BasedCollaborativeFilteringBoLi\u2217VanderbiltUniversitybo.li.2@vanderbilt.eduYiningWang\u2217CarnegieMellonUniversityynwang.yining@gmail.comAartiSinghCarnegieMellonUniversityaarti@cs.cmu.eduYevgeniyVorobeychikVanderbiltUniversityyevgeniy.vorobeychik@vanderbilt.eduAbstractRecommendationandcollaborative\ufb01lteringsystemsareimportantinmoderninfor-mationande-commerceapplications.Asthesesystemsarebecomingincreasinglypopularintheindustry,theiroutputscouldaffectbusinessdecisionmaking,in-troducingincentivesforanadversarialpartytocompromisetheavailabilityorintegrityofsuchsystems.Weintroduceadatapoisoningattackoncollaborative\ufb01lteringsystems.Wedemonstratehowapowerfulattackerwithfullknowledgeofthelearnercangeneratemaliciousdatasoastomaximizehis/hermaliciousobjectives,whileatthesametimemimickingnormaluserbehaviortoavoidbeingdetected.Whilethecompleteknowledgeassumptionseemsextreme,itenablesarobustassessmentofthevulnerabilityofcollaborative\ufb01lteringschemestohighlymotivatedattacks.Wepresentef\ufb01cientsolutionsfortwopopularfactorization-basedcollaborative\ufb01lteringalgorithms:thealternativeminimizationformulationandthenuclearnormminimizationmethod.Finally,wetesttheeffectivenessofourproposedalgorithmsonreal-worlddataanddiscusspotentialdefensivestrategies.1IntroductionRecommendationsystemshaveemergedasacrucialfeatureofmanyelectroniccommercesystems.Inmachinelearningsuchproblemsareusuallyreferredtoascollaborative\ufb01lteringormatrixcompletion,wheretheknownusers\u2019preferencesareabstractedintoanincompleteuser-by-itemmatrix,andthegoalistocompletethematrixandsubsequentlymakenewitemrecommendationsforeachuser.Existingapproachesintheliteratureincludenearest-neighbormethods,whereauser\u2019s(item\u2019s)preferenceisdeterminedbyotherusers(items)withsimilarpro\ufb01les[1],andfactorization-basedmethodswheretheincompletepreferencematrixisassumedtobeapproximatelylow-rank[2,3].Asrecommendationsystemsplayaneverincreasingroleincurrentinformationande-commercesystems,theyaresusceptibletoariskofbeingmaliciouslyattacked.Oneparticularformofattacksiscalleddatapoisoning,inwhichamaliciouspartycreatesdummy(malicious)usersinarecom-mendationsystemwithcarefullychosenitempreferences(i.e.,data)suchthattheeffectivenessorcredibilityofthesystemismaximallydegraded.Forexample,anattackermightattempttomakerecommendationsthatareasdifferentaspossiblefromthosethatwouldotherwisebemadebytherecommendationsystem.Inanother,moresubtle,example,theattackerisassociatedwiththeproducerofaspeci\ufb01cmovieorproduct,whomaywishtoincreaseordecreasethepopularityofacertainitem.Inbothcases,thecredibilityofarecommendationsystemisharmedbythemaliciousactivities,whichcouldleadtosigni\ufb01canteconomicloss.Duetotheopennatureofrecommendation\u2217Bothauthorscontributeequally30thConferenceonNeuralInformationProcessingSystems(NIPS2016),Barcelona,Spain.\fsystemsandtheirrelianceonuser-speci\ufb01edjudgmentsforbuildingpro\ufb01les,variousformsofattacksarepossibleandhavebeendiscussed,suchastherandomattackandrandomproductpush/nukeattack[4,5].However,theseattacksarenotformallyanalyzedandcannotbeoptimizedaccordingtospeci\ufb01ccollaborative\ufb01lteringalgorithms.Asitisnotdif\ufb01cultforattackerstodeterminethedefender\u2019s\ufb01lteringalgorithmorevenitsparameterssettings(e.g.,throughinsiderattacks),thiscanleadonetosigni\ufb01cantlyunder-estimatetheattacker\u2019sabilityandresultinsubstantialloss.Wepresentasystematicapproachtocomputingnear-optimaldatapoisoningattacksforfactorization-basedcollaborative\ufb01ltering/recommendationmodels.WeassumeahighlymotivatedattackerwithknowledgeofboththelearningalgorithmsandparametersofthelearnerfollowingtheKerckhoffs\u2019principletoensurereliablevulnerabilityanalysisintheworstcase.Wefocusontwomostpopularalgorithms:alternatingminimization[6]andnuclearnormminimization[3].Ourmaincontributionsareasfollows:\u2022Comprehensivecharacterizationofattackerutilities:Wecharacterizeseveralattackerutilities,whichincludeavailabilityattacks,wherepredictionerrorisincreased,andintegrityattacks,whereitem-speci\ufb01cobjectivesareconsidered.Optimalattackstrategiesforallutilitiescanbecomputedunderauni\ufb01edoptimizationframework.\u2022Novelgradientcomputations:Buildinguponexistinggradient-baseddatapoisoningframe-works[7,8,9],wedevelopnovelmethodsforgradientcomputationbasedon\ufb01rst-orderKKTconditionsfortwowidelyusedalgorithms:alternatingminimization[6]andnuclearnormminimization[2].Theresultingderivationsarehighlynon-trivial;inaddition,toourknowledgethisworkisthe\ufb01rsttogivesystematicdatapoisoningattacksforproblemsinvolvingnon-smoothnuclearnormtypeobjectives.\u2022Mimickingnormaluserbehaviors:Fordatapoisoningattacks,mostpriorworkfocusesonmaximizingattacker\u2019sutility.Alessinvestigatedproblemishowtosynthesizemaliciousdatapointsthatarehardforadefendertodetect.InthispaperweprovideanoveltechniquebasedonstochasticgradientLangevindynamicsoptimization[10]toproducemalicioususersthatmimicnormaluserbehaviorsinordertoavoiddetection,whileachievingattackobjectives.RelatedWork:Therehasbeenextensivepriorresearchconcerningthesecurityofmachinelearningalgorithms[11,12,13,14,15].Biggioetal.pioneeredtheresearchofoptimizingmaliciousdata-drivenattacksforkernel-basedlearningalgorithmssuchasSVM[16].Thekeyoptimizationtechniqueistoapproximatelycomputeimplicitgradientsofthesolutionofanoptimizationproblembasedon\ufb01rst-orderKKTconditions.Similartechniqueswerelatergeneralizedtooptimizedatapoisoningattacksforseveralotherimportantlearningalgorithms,suchasLassoregression[7],topicmodeling[8],andautoregressivemodels[17].Thereadermayreferto[9]forageneralalgorithmicframeworkoftheabovementionedmethods.Intermsofcollaborative\ufb01ltering/matrixcompletion,thereisanotherlineofestablishedresearchthatfocusesonrobustmatrixcompletion,inwhichasmallportionofelementsorrowsintheunderlyinglow-rankmatrixisassumedtobearbitrarilyperturbed[18,19,20,21].Speci\ufb01cally,thestabilityofalternatingminimizationsolutionswasanalyzedwithrespecttomaliciousdatamanipulationsin[22].However,[22]assumesagloballyoptimalsolutionofalternatingminimizationcanbeobtained,whichisrarelytrueinpractice.2PreliminariesWe\ufb01rstsetupthecollaborative\ufb01ltering/matrixcompletionproblemandgiveanoverviewofexistinglow-rankfactorizationbasedapproaches.LetM\u2208Rm\u00d7nbeadatamatrixconsistingofmrowsandncolumns.Mijfori\u2208[m]andj\u2208[n]wouldthencorrespondtotheratingtheithusergivesforthejthitem.Weuse\u2126={(i,j):Mijisobserved}todenoteallobservableentriesinMandassumethat|\u2126|(cid:28)mn.Wealsouse\u2126i\u2286[n]and\u21260j\u2286[m]forcolumns(rows)thatareobservableattheithrow(jthcolumn).Thegoalofcollaborative\ufb01ltering(alsoreferredtoasmatrixcompletioninthestatisticallearningliterature[2])isthentorecoverthecompletematrixMfromfewobservationsM\u2126.Thematrixcompletionproblemisingeneralill-posedasitisimpossibletocompleteanarbitrarymatrixwithpartialobservations.Asaresult,additionalassumptionsareimposedontheunderlyingdatamatrixM.OnestandardassumptionisthatMisveryclosetoanm\u00d7nrank-kmatrixwith2\fk(cid:28)min(m,n).Undersuchassumptions,thecompletematrixMcanberecoveredbysolvingthefollowingoptimizationproblem:minX\u2208Rm\u00d7nkR\u2126(M\u2212X)k2F,s.t.rank(X)\u2264k,(1)wherekAk2F=Pi,jA2ijdenotesthesquaredFrobeniousnormofmatrixAand[R\u2126(A)]ijequalsAijif(i,j)\u2208\u2126and0otherwise.Unfortunately,thefeasiblesetinEq.(1)isnon-convex,makingtheoptimimzationproblemdif\ufb01culttosolve.Therehasbeenanextensivepriorliteratureonap-proximatelysolvingEq.(1)and/oritssurrogatesthatleadtotwostandardapproaches:alternatingminimizationandnuclearnormminimization.Forthe\ufb01rstapproach,oneconsidersthefollowingproblem:minU\u2208Rm\u00d7k,V\u2208Rn\u00d7k(cid:8)kR\u2126(M\u2212UV>)k2F+2\u03bbUkUk2F+2\u03bbVkVk2F(cid:9).(2)Eq.(2)isequivalenttoEq.(1)when\u03bbU=\u03bbV=0.Inpracticepeopleusuallysetbothregularizationparameters\u03bbUand\u03bbVtobesmallpositiveconstantsinordertoavoidlargeentriesinthecompletedmatrixandalsoimproveconvergence.SinceEq.(2)isbi-convexinUandV,analternatingminimizationprocedurecanbeapplied.Alternatively,onesolvesanuclear-normminimizationproblemminX\u2208Rm\u00d7nkR\u2126(M\u2212X)k2F+2\u03bbkXk\u2217,(3)where\u03bb>0isaregularizationparameterandkXk\u2217=Prank(X)i=1|\u03c3i(X)|isthenuclearnormofX,whichactsasaconvexsurrogateoftherankfunction.Eq.(3)isaconvexoptimizationfunctionandcanbesolvedusinganiterativesingularvaluethresholdingalgorithm[3].ItcanbeshownthatbothmethodsinEq.(2)and(3)provablyapproximatethetrueunderlyingdatamatrixMundercertainconditions[6,2].3TheAttackModelInthissectionwedescribethedatapoisoningattackmodelconsideredinthispaper.Foradatamatrixconsistingofmusersandnitems,theattackeriscapableofadding\u03b1mmalicioususerstothetrainingdatamatrix,andeachmalicioususerisallowedtoreporthis/herpreferenceonatmostBitemswitheachpreferenceboundedintherange[\u2212\u039b,\u039b].Beforeproceedingtodescribetheattacker\u2019sgoals,we\ufb01rstintroducesomenotationtofacilitatepresentation.WeuseM\u2208Rm\u00d7ntodenotetheoriginaldatamatrixandfM\u2208Rm0\u00d7ntodenotethedatamatrixofallm0=\u03b1mmalicioususers.Lete\u2126bethesetofnon-zeroentriesinfMande\u2126i\u2286[n]beallitemsthattheithmalicioususerrated.Accordingtoourattackmodels,|e\u2126i|\u2264Bforeveryi\u2208{1,\u00b7\u00b7\u00b7,m0}andkfMkmax=max|fMij|\u2264\u039b.Let\u0398\u03bb(fM;M)betheoptimalsolutioncomputedjointlyontheoriginalandpoisoneddatamatrices(fM;M)usingregularizationparameters\u03bb.Forexample,Eq.(2)becomes\u0398\u03bb(fM;M)=argminU,eU,VkR\u2126(M\u2212UV>)k2F+kR\u02dc\u2126(fM\u2212eUV>)k2F+2\u03bbU(kUk2F+keUk2F)+2\u03bbVkVk2F(4)wheretheresulting\u0398consistsoflow-ranklatentfactorsU,eUfornormalandmalicioususersaswellasVforitems.Simiarly,forthenuclearnormminimizationformulationinEq.(3),wehave\u0398\u03bb(fM;M)=argminX,eXkR\u2126(M\u2212X)k2F+kR\u02dc\u2126(fM\u2212eX)k2F+2\u03bbk(X;eX)k\u2217,(5)where\u0398=(X,eX).LetcM(\u0398)bethematrixestimatedfromlearntmodel\u0398.Forexample,forEq.(4)wehavecM(\u0398)=UV>andforEq.(5)wehavecM(\u0398)=X.Thegoaloftheattackeristo\ufb01ndoptimalmalicioususersfM\u2217suchthatfM\u2217\u2208argmaxfM\u2208MR(cM(\u0398\u03bb(fM;M)),M).(6)HereM={fM\u2208Rm0\u00d7n:|\u02dc\u2126i|\u2264B,kfMkmax\u2264\u039b}isthesetofallfeasiblepoisoningattacksdiscussedearlierinthissectionandR(cM,M)denotestheattacker\u2019sutilityfordivertingthecollabo-rative\ufb01lteringalgorithmtopredictcMonanoriginaldatasetM,withthehelpoffewmalicioususersfM.Belowwelistseveraltypicalattackerutilities:3\fAvailabilityattacktheattackerwantstomaximizetheerrorofthecollaborative\ufb01lteringsystem,andeventuallyrenderthesystemuseless.SupposeMisthepredictionofthecollaborative\ufb01lteringsystemwithoutdatapoisoningattacks.2Theutilityfunctionisthende\ufb01nedasthetotalamountofperturbationofpredictionsbetweenMandcM(predictionsafterpoisoningattacks)onunseenentries\u2126C:Rav(cM,M)=kR\u2126C(cM\u2212M)k2F.(7)Integrityattackinthismodeltheattackerwantstoboost(orreduce)thepopularityofa(subset)ofitems.SupposeJ0\u2286[n]isthesubsetofitemstheattackerisinterestedinandw:J0\u2192Risapre-speci\ufb01edweightvectorbytheattacker.TheutilityfunctionisRinJ0,w(cM,M)=mXi=1Xj\u2208J0w(j)cMij.(8)Hybridattackahybridlossfunctioncanalsobede\ufb01ned:RhybridJ0,w,\u00b5(cM,M)=\u00b51RavJ0,w(cM,M)+\u00b52Rin(cM,M),(9)where\u00b5=(\u00b51,\u00b52)arecoef\ufb01cientsthattradeofftheavailabilityandintegrityattackobjectives.Inaddition,\u00b51couldbenegative,whichmodelsthecasewhentheattackerwantstoleavea\u201clighttrace\":theattackerwantstomakehisitemmorepopularwhilemakingtheotherrecommendationsofthesystemlessperturbedtoavoiddetection.4ComputingOptimalAttackStrategiesWedescribepracticalalgorithmstosolvetheoptimizationprobleminEq.(6)foroptimalattackstrategyfM\u2217thatmaximizestheattacker\u2019sutility.We\ufb01rstconsiderthealternatingminimizationformulationinEq.(4)andderiveaprojectedgradientascentmethodthatsolvesforthecorrespondingoptimalattackstrategy.SimilarderivationsarethenextendedtothenuclearnormminimizationformulationinEq.(5).Finally,wediscusshowtodesignmalicioususersthatmimicnormaluserbehaviorinordertoavoiddetection.4.1AttackingAlternatingMinimizationWeusetheprojectedgradientascent(PGA)methodforsolvingtheoptimizationprobleminEq.(6)withrespecttothealternatingminimizationformulationinEq.(4):initerationtweupdatefM(t)asfollows:fM(t+1)=ProjM(cid:16)fM(t)+st\u00b7\u2207fMR(cM,M)(cid:17),(10)whereProjM(\u00b7)istheprojectionoperatorontothefeasibleregionMandstisthestepsizeiniterationt.NotethattheestimatedmatrixcMdependsonthemodel\u0398\u03bb(fM;M)learntonthejointdatamatrix,whichfurtherdependsonthemalicioususersfM.SincetheconstraintsetMishighlynon-convex,wegenerateBitemsuniformlyatrandomforeachmalicioususertorate.TheProjM(\u00b7)operatorthenreducestoprojectingeachmalicioususers\u2019ratingvectorontoan\u2018\u221eballofdiameter\u039b,whichcanbeeasilyevaluatedbytruncatingallentriesinfMatthelevelof\u00b1\u039b.Wenextshowhowto(approximately)compute\u2207fMR(cM,M).Thisischallengingbecauseoneoftheargumentsinthelossfunctioninvolvesanimplicitoptimizationproblem.We\ufb01rstapplychainruletoarriveat\u2207fMR(cM,M)=\u2207fM\u0398\u03bb(fM;M)\u2207\u0398R(cM,M).(11)Thesecondgradient(withrespectto\u0398)iseasytoevaluate,asalllossfunctionsmentionedintheprevioussectionaresmoothanddifferentiable.Detailedderivationof\u2207\u0398R(cM,M)isdeferredtoAppendixA.Ontheotherhand,the\ufb01rstgradienttermtermismuchhardertoevaluatebecause\u0398\u03bb(\u00b7)isanoptimizationprocedure.Inspiredby[7,8,9],weexploittheKKTconditionsoftheoptimizationproblem\u0398\u03bb(\u00b7)toapproximatelycompute\u2207fM\u0398\u03bb(fM;M).Morespeci\ufb01cally,theoptimalsolution\u0398=(U,eU,V)ofEq.(4)satis\ufb01es\u03bbUui=Xj\u2208\u2126i(Mij\u2212u>ivj)vj;2Notethatwhenthecollaborative\ufb01lteringalgorithmanditsparametersareset,MisafunctionofobservedentriesR\u2126(M).4\fAlgorithm1OptimizingfMviaPGA1:Input:Originalpartiallyobservedm\u00d7ndatamatrixM,algorithmregularizationparameter\u03bb,attackbudgetparameters\u03b1,Band\u039b,attacker\u2019sutilityfunctionR,stepsize{st}\u221et=1.2:Initialization:randomfM(0)\u2208Mwithbothratingsandrateditemsuniformlysampledatrandom;t=0.3:whilefM(t)doesnotconvergedo4:Computetheoptimalsolution\u0398\u03bb(fM(t);M).5:Computegradient\u2207fMR(cM,M)usingEq.(10).6:Update:fM(t+1)=ProjM(fM(t)+st\u2207fMR).7:t\u2190t+1.8:endwhile9:Output:m0\u00d7nmaliciousmatrixfM(t).\u03bbU\u02dcui=Xj\u2208e\u2126i(fMij\u2212\u02dcu>ivj)vj;\u03bbVvj=Xi\u2208\u21260j(Mij\u2212u>ivj)ui+Xi\u2208e\u21260j(fMij\u2212\u02dcu>ivj)\u02dcui,whereui,\u02dcuiaretheithrows(ofdimensionk)inUoreUandvjisthejthrow(alsoofdimensionk)inV.Subsequently,{ui,\u02dcui,vj}canbeexpressedasfunctionsoftheoriginalandmaliciousdatamatricesMandfM.Usingthefactthat(a>x)a=(aa>)xandMdoesnotchangewithfM,weobtain\u2202ui(fM)\u2202fMij=0;\u2202\u02dcui(fM)\u2202fMij=(cid:16)\u03bbUIk+\u03a3(i)U(cid:17)\u22121vj;\u2202vj(fM)\u2202fMij=(cid:16)\u03bbVIk+\u03a3(j)V(cid:17)\u22121ui.Here\u03a3(i)Uand\u03a3(j)Varede\ufb01nedas\u03a3(i)U=Xj\u2208\u2126i\u222ae\u2126ivjv>j,\u03a3(j)V=Xi\u2208\u21260j\u222ae\u21260juiu>i.(12)AframeworkoftheproposedoptimizationalgorithmisdescribedinAlgorithm1.4.2AttackingNuclearNormMinimizationWeextendtheprojectedgradientascentalgorithminSec.4.1tocomputeoptimalattackstrategiesforthenuclearnormminimizationformulationinEq.(5).SincetheobjectiveinEq.(5)isconvex,theglobaloptimalsolution\u0398=(X,eX)canbeobtainedbyconventionalconvexoptimizationproceduressuchasproximalgradientdescent(a.k.a.singularvaluethresholding[3]fornuclearnormminimization).Inaddition,theresultingestimation(X;eX)islowrankduetothenuclearnormpenalty[2].Suppose(X;eX)hasrank\u03c1\u2264min(m,n).Weuse\u03980=(U,eU,V,\u03a3)asanalternativecharacterizationofthelearntmodelwithareducednumberofparameters.HereX=U\u03a3V>andeX=eU\u03a3V>aresingularvaluedecompositionsofXandeX;thatis,U\u2208Rm\u00d7\u03c1,eU\u2208Rm0\u00d7\u03c1,V\u2208Rn\u00d7\u03c1haveorthornormalcolumnsand\u03a3=diag(\u03c31,\u00b7\u00b7\u00b7,\u03c3\u03c1)isanon-negativediagonalmatrix.Tocomputethegradient\u2207fMR(cM,M),weagainapplythechainruletodecomposethegradientintotwoparts:\u2207fMR(cM,M)=\u2207fM\u03980\u03bb(fM;M)\u2207\u03980R(cM,M).(13)SimilartoEq.(11),thesecondgradientterm\u2207\u03980R(cM,M)isrelativelyeasiertoevaluate.ItsderivationdetailsaredeferredtotheAppendix.Intheremainderofthissectionweshallfocusonthecomputationofthe\ufb01rstgradientterm,whichinvolvespartialderivativesof\u03980=(U,eU,V,\u03a3)withrespecttomalicioususersfM.WebeginwiththeKKTconditionattheoptimalsolution\u03980ofEq.(5).Unlikethealternatingminimizationformulation,thenuclearnormfunctionk\u00b7k\u2217isnoteverywheredifferentiable.Asa5\fAlgorithm2OptimizingfMviaSGLD1:Input:Originalpartiallyobservedm\u00d7ndatamatrixM,algorithmregularizationparameter\u03bb,attackbudgetparameters\u03b1,Band\u039b,attacker\u2019sutilityfunctionR,stepsize{st}\u221et=1,tuningparameter\u03b2,numberofSGLDiterationsT.2:Priorsetup:compute\u03bej=1mPmi=1Mijand\u03c32j=1mPmi=1(Mij\u2212\u03bej)2foreveryj\u2208[n].3:Initialization:samplefM(0)ij\u223cN(\u03bej,\u03c32j)fori\u2208[m0]andj\u2208[n].4:fort=0toTdo5:Computetheoptimalsolution\u0398\u03bb(fM(t);M).6:Computegradient\u2207fMR(cM,M)usingEq.(10).7:UpdatefM(t+1)accordingtoEq.(17).8:endfor9:Projection:\ufb01ndfM\u2217\u2208argminfM\u2208MkfM\u2212fM(t)k2F.Detailsinthemaintext.10:Output:m0\u00d7nmaliciousmatrixfM\u2217.result,theKKTconditionrelatesthesubdifferentialofthenuclearnormfunction\u2202k\u00b7k\u2217asR\u2126,\u02dc\u2126(cid:16)[M;fM]\u2212[X;eX](cid:17)\u2208\u03bb\u2202k[X;eX]k\u2217.(14)Here[X;eX]istheconcatenated(m+m0)\u00d7nmatrixofXandeX.Thesubdifferentialofthenuclearnormfunction\u2202k\u00b7k\u2217isalsoknown[2]:\u2202kXk\u2217=nUV>+W:U>W=WV=0,kWk2\u22641o,whereX=U\u03a3V>isthesingularvaluedecompositionofX.Suppose{ui},{\u02dcui}and{vj}arerowsofU,eU,VandW={wij}.Wecanthenre-formulatetheKKTconditionEq.(14)asfollows:\u2200(i,j)\u2208\u2126,Mij=u>i(\u03a3+\u03bbI\u03c1)vj+\u03bbwij;\u2200(i,j)\u2208e\u2126,fMij=\u02dcu>i(\u03a3+\u03bbI\u03c1)vj+\u03bb\u02dcwij.Nowwederive\u2207fM\u0398=\u2207fM(u,\u02dcu,v,\u03c3);thefullderivationisdeferredtotheextendedversion3.4.3MimicingNormalUserBehaviorsNormalusersgenerallydonotrateitemsuniformlyatrandom.Forexample,somemoviesaresigni\ufb01cantlymorepopularthanothers.Asaresult,malicioususersthatpickratedmoviesuniformlyatrandomcanbeeasilyidenti\ufb01edbyrunningat-testagainstaknowndatabaseconsistingofonlynormalusers,asshowninSec.5.Toalleviatethisissue,inthissectionweproposeanalternativeapproachtocomputedatapoisoningattackssuchthattheresultingmalicioususersfMmimicsnormalusersMtoavoidpotentialdetection,whilestillachievingreasonablylargeutilityR(cM,M)fortheattacker.WeuseaBayesianformulationtotakebothdatapoisoninganddetectionavoidanceobjectivesintoconsideration.Thepriordistributionp0(fM)capturesnormaluserbehaviorsandisde\ufb01nedasamultivariatenormaldistributionp0(fM)=m0Yi=1nYj=1N(fMij;\u03bej,\u03c32j),where\u03bejand\u03c32jaremeanandvarianceparametersfortheratingofthejthitemprovidedbynormalusers.InpracticebothparameterscanbeestimatedusingnormalusermatrixMas\u03bej=1mPmi=1Mijand\u03c32=1mPmi=1(Mij\u2212\u03bej)2.Ontheotherhand,thelikelihoodp(M|fM)isde\ufb01nedasp(M|fM)=1Zexp(cid:16)\u03b2\u00b7R(cM,M)(cid:17),(15)whereR(cM,M)=R(cM(\u0398\u03bb(fM;M)),M)isoneoftheattackerutilityfunctionsde\ufb01nedinSec.3,Zisanormalizationconstantand\u03b2>0isatuningparameterthattradesoffattackperformanceand3http://arxiv.org/abs/1608.081826\f(a)(b)(c)(d)Figure1:RMSE/Averageratingsforalternatingminimizationwithdifferentpercentageofmaliciouspro\ufb01les;(a)\u00b51=1,\u00b52=0,(b)\u00b51=1,\u00b52=\u22121,(c)\u00b51=0,\u00b52=1,(d)\u00b51=\u22121,\u00b52=1.detectionavoidance.Asmall\u03b2shiftstheposterioroffMtowarditsprior,whichmakestheresultingattackstrategylesseffectivebuthardertodetect,andviceversa.Givenbothpriorandlikelihoodfunctions,aneffectivedetection-avoidingattackstrategyfMcanbeobtainedbysamplingfromitsposteriordistribution:p(fM|M)=p0(fM)p(M|fM)/p(M)\u221dexp\uf8eb\uf8ed\u2212m0Xi=1nXj=1(fMij\u2212\u03bej)22\u03c32j+\u03b2R(cM,M)\uf8f6\uf8f8.(16)PosteriorsamplingofEq.(16)isclearlyintractableduetotheimplicitandcomplicateddependencyoftheestimatedmatrixcMonthemaliciousdatafM,thatis,cM=cM(\u0398\u03bb(fM;M))).Tocircumventthisproblem,weapplyStochasticGradientLangevinDynamics(SGLD,[10])toapproximatelysamplefMfromitsposteriordistributioninEq.(16).Morespec\ufb01cally,theSGLDalgorithmiterativelycomputesasequenceofposteriorsamples{fM(t)}t\u22650andiniterationtthenewsamplefM(t+1)iscomputedasfM(t+1)=fM(t)+st2(cid:16)\u2207fMlogp(fM|M)(cid:17)+\u03b5t,(17)where{st}t\u22650arestepsizesand\u03b5t\u223cN(0,stI)areindependentGaussiannoisesinjectedateachSGLDiteration.Thegradient\u2207fMlogp(fM|M)canbecomputedas\u2207fMlogp(fM|M)=\u2212(fM\u2212\u039e)\u03a3\u22121+\u03b2\u2207fMR(cM,M),where\u03a3=diag(\u03c321,\u00b7\u00b7\u00b7,\u03c32n)and\u039eisanm0\u00d7nmatrixwith\u039eij=\u03bejfori\u2208[m0]andj\u2208[n].Theothergradient\u2207fMR(cM,M)canbecomputedusingtheprocedureinSections4.1and4.2.Finally,thesampledmaliciousmatrixfM(t)isprojectedbackontothefeasiblesetMbyselectingBitemsperuserwiththelargestabsoluteratingandtruncatingratingstothelevelof{\u00b1\u039b}.Ahigh-leveldescriptionoftheproposedmethodisgiveninAlgorithm2.5ExperimentalResultsToevaluatetheeffectivenessofourproposedpoisoningattackstrategy,weusethepubliclyavailableMovieLensdatasetwhichcontains20millionsratingsand465,000tagapplicationsappliedto27,000moviesby138,000users[23].Weshifttheratingrangeto[\u22122,2]forcomputationconvenience.Toavoidthe\u201ccold-start\u201dproblem,weconsideruserswhohaveratedatleast20movies.Twometricsareemployedtomeasuretherelativeperformanceofthesystemsbeforeandafterdatapoisoningattacks:rootmeansquareerror(RMSE)forthepredictedunseenentries4andaverageratingforspeci\ufb01citems.Wethenanalyzethetradeoffbetweenattackperformanceanddetectionavoidance,whichiscontroledbythe\u03b2parameterinEq.(15).Thisservesasaguideforhow\u03b2shouldbesetinlaterexperiments.Weuseapairedt-testtocomparethedistributionsofrateditemsbetweennormalandmalicioususers.Wepresentthetrendofp-valueagainstdifferentvaluesof\u03b2intheextendedversionofthepaper.Tostriveforagoodtradeoff,weset\u03b2=0.6atwhichthep-valuestablizesaround0.7andthepoisoningattackperformanceisnotsigni\ufb01cantlysacri\ufb01ced.Weemployattackmodelsspeci\ufb01edinEq.(9),wheretheutilityparameters\u00b51and\u00b52balancetwodifferentmaliciousgoals(availabilityandintegrity)anattackerwishestoachieve.Fortheintegrity4de\ufb01nedasRMSE=qP(i,j)\u2208\u2126C(Mij\u2212cMij)2/|\u2126C|,whereMisthepredictionofmodeltrainedoncleandataR\u2126(M)only(i.e.,withoutdatapoisoningattacks).7\f(a)(b)(c)(d)Figure2:RMSE/Averageratingsfornuclearnormminimizationwithdifferentpercentageofmali-ciouspro\ufb01les;(a)\u00b51=1,\u00b52=0,(b)\u00b51=1,\u00b52=\u22121,(c)\u00b51=0,\u00b52=1,(d)\u00b51=\u22121,\u00b52=1.utilityRinJ0,w,theJ0setcontainsonlyoneitemj0selectedrandomlyfromallitemswhoseaveragepredictedratingsarearound0.8.Theweightwj0issetaswj0=2.Figure1(a)(b)plotstheRMSEafterdatapoisoningattacks.When\u00b51=1,\u00b52=0,theattackerisinterestedinincreasingtheRMSEofthecollaborative\ufb01lteringsystemandhencereducingthesystem\u2019savailability.Ontheotherhand,when\u00b51=1,\u00b52=\u22121theattackerwishestoincreaseRMSEwhileatthesametimekeepingtheratingofspeci\ufb01citems(j0)aslowaspossibleforcertainmaliciouspurposes.Figure1(b)showsthatwhentheattackersconsidertobothobjectives(\u00b51=1,\u00b52=\u22121),theRMSEafterpoisoningisslightlylowerthanthatifonlyavailabilityistargeted(\u00b51=1,\u00b52=0).Inaddition,theprojectedgradientascent(PGA)strategygeneratesthelargestRMSEscorecomparedwiththeothermethods.However,PGArequiresmalicioususerstorateeachitemuniformlyatrandom,whichmightexposethemaliciouspro\ufb01lestoaninformeddefender.Morespeci\ufb01cally,thepairedt-testonthosemaliciouspro\ufb01lesproducedbyPGArejectsthenullhypothesisthattheitemsratedbytheattackerstrategiesarethesameasthoseobtainedfromnormalusers(p<0.05).Incontrast,theSGLDmethodleadstoslightlyworseattackerutilitybutgeneratesmalicioususersthatarehardtodistinguishfromthenormalusers(forexample,thepairedt-testleadstoinconclusivep-values(largerthan0.7)with\u03b2=0.6.Finally,bothPGAandSGLDresultinhigherattackerutilitycomparedtouniformattacks,wherebothratingsandrateditemsaresampleduniformlyatrandomformaliciouspro\ufb01les.ApartfromtheRMSEscores,wealsoplotratingsofspeci\ufb01citemsagainstpercentageofmaliciouspro\ufb01lesinFigure1(c)(d).Weconsidertwoadditionalattackutilitysettings:\u00b51=0,\u00b52=1,inwhichtheattackerwishestopushtheratingsofsomeparticularitems(speci\ufb01edinwandJ0ofRin)ashighaspossible;and\u00b51=\u22121,\u00b52=1,wheretheattackeralsowantstoleavea\u201clighttrace\"byreducingtheimpactontheentiresystemresultedfrommaliciousactivities.Itisclearthattargetedattackes(bothPGAandSGLD)areindeedmoreeffectiveatmanipulatingratingsofspeci\ufb01citemsforintegrityattacks.WealsoplotRMSE/AverageratingsagainstmalicioususerpercentageinFigure2forthenuclearnormminimizationundersimilarsettingsbasedonasubsetof1000usersand1700movies(items),sinceitismorecomputationallyexpensivethanalternatingminimization.Ingeneral,weobservesimilarbehaviorofbothRMSE/Averageratingsunderdifferentattackingmodels\u00b51,\u00b52withalternatingminimization.6DiscussionandConcludingRemarksOurultimategoalforthepoisoningattackanalysisistodeveloppossibledefensivestrategiesbasedonthecarefulanalysisofadversarialbehaviors.Sincethepoisoningdataisoptimizedbasedontheattacker\u2019smaliciousobjectives,thecorrelationsamongfeatureswithinafeaturevectormaychangetoappeardifferentfromnormalinstances.Therefore,trackinganddetectingdeviationsinthefeaturecorrelationsandotheraccuracymetricscanbeonepotentialdefense.Additionally,defendercanalsoapplythecombinationalmodelsorsamplingstrategies,suchasbagging,toreducethein\ufb02uenceofpoisoningattacks.AcknowledgmentsThisresearchwaspartiallysupportedbytheNSF(CNS-1238959,IIS-1526860),ONR(N00014-15-1-2621),ARO(W911NF-16-1-0069),AFRL(FA8750-14-2-0180),SandiaNationalLaboratories,andSymantecLabsGraduateResearchFellowship.8\fReferences[1]JunWang,ArjendeVires,andMarcelReinders.Unifyinguser-basedanditem-basedcollaborative\ufb01lteringapproachesbysimilarityfusion.InSIGIR,2006.[2]EmmanuelCand\u00e8sandBenRecht.Exactmatrixcompletionviaconvexoptimization.FoundationsofComputationalMathematics,9(6):717\u2013772,2007.[3]Jian-FengCai,EmmanuelCand\u00e8s,andZuoweiShen.Asingularvaluethresholdingalgorithmformatrixcompletion.SIAMJournalonOptimization,20(4):1956\u20131982,2010.[4]BamshadMobasher,RobinBurke,RunaBhaumik,andChadWilliams.Effectiveattackmodelsforshillingitem-basedcollaborative\ufb01lteringsystems.InProceedingsofthe2005WebKDDWorkshop,heldinconjuctionwithACMSIGKDD\u20192005,2005.[5]MichaelPO\u2019Mahony,NeilJHurley,andGuenoleCMSilvestre.Promotingrecommendations:Anattackoncollaborative\ufb01ltering.InDatabaseandExpertSystemsApplications,pages494\u2013503.Springer,2002.[6]PrateekJain,PraneethNetrapalli,andSujaySanghavi.Low-rankmatrixcompletionusingalternatingminimization.InSTOC,2013.[7]HuangXiao,BattistaBiggio,GavinBrown,GiorgioFumera,ClaudiaEckert,andFabioRoli.Isfeatureselectionsecureagainsttrainingdatapoisoning.InICML,2015.[8]ShikeMeiandXiaojinZhu.Thesecurityoflatentdirichletallocation.InAISTATS,2015.[9]ShikeMeiandXiaojinZhu.Usingmachineteachingtoidentifyoptimaltraining-setattacksonmachinelearners.InAAAI,2015.[10]MaxWellingandYeeWTeh.Bayesianlearningviastochasticgradientlangevindynamics.InProceedingsofthe28thInternationalConferenceonMachineLearning(ICML-11),pages681\u2013688,2011.[11]NileshDalvi,PedroDomingos,SumitSanghai,DeepakVerma,etal.Adversarialclassi\ufb01cation.InProceedingsofthetenthACMSIGKDDinternationalconferenceonKnowledgediscoveryanddatamining,pages99\u2013108.ACM,2004.[12]DanielLowdandChristopherMeek.Adversariallearning.InProceedingsoftheeleventhACMSIGKDDinternationalconferenceonKnowledgediscoveryindatamining,pages641\u2013647.ACM,2005.[13]BoLiandYevgeniyVorobeychik.Featurecross-substitutioninadversarialclassi\ufb01cation.InAdvancesinNeuralInformationProcessingSystems,pages2087\u20132095,2014.[14]BoLiandYevgeniyVorobeychik.Scalableoptimizationofrandomizedoperationaldecisionsinadversarialclassi\ufb01cationsettings.InProceedingsoftheEighteenthInternationalConferenceonArti\ufb01cialIntelligenceandStatistics,pages599\u2013607,2015.[15]MarcoBarreno,BlaineNelson,RussellSears,AnthonyDJoseph,andJDougTygar.Canmachinelearningbesecure?InProceedingsofthe2006ACMSymposiumonInformation,computerandcommunicationssecurity,pages16\u201325.ACM,2006.[16]BattistaBiggio,BlaineNelson,andPavelLaskov.Poisoningattacksagainstsupportvectormachines.InICML,2012.[17]ScottAlfeld,XiaojinZhu,andPaulBarford.Datapoisoningattacksagainstautoregressivemodels.InAAAI,2016.[18]OlgaKlopp,KarimLounici,andAlexandreTsybakov.Robustmatrixcompletion.arXiv:1412.8132,2014.[19]YudongChen,HuanXu,ConstantineCaramanis,andSujaySanghavi.Robustmatrixcompletionandcorruptedcolumns.InICML,2011.[20]YudongChen,AliJalali,SujaySanghavi,andConstantineCaramanis.Low-rankmatrixrecoveryfromerrorsanderasures.IEEETransactionsonInformationTheory,59(7):4324\u20134337,2013.[21]FeipingNie,HuaWang,XiaoCai,HengHuang,andChrisDing.Robustmatrixcompletionviajointschattenp-normandlp-normminimization.InICDM,2012.[22]Yu-XiangWangandHuanXu.Stabilityofmatrixfactorizationforcollaborative\ufb01ltering.InICML,2012.[23]ResearchGroupLens.www.grouplens.org.9\f", "award": [], "sourceid": 1036, "authors": [{"given_name": "Bo", "family_name": "Li", "institution": "Vanderbilt University"}, {"given_name": "Yining", "family_name": "Wang", "institution": "Carnegie Mellon University"}, {"given_name": "Aarti", "family_name": "Singh", "institution": "Carnegie Mellon University"}, {"given_name": "Yevgeniy", "family_name": "Vorobeychik", "institution": "Vanderbilt University"}]}