{"title": "Multi-effect Decompositions for Financial Data Modeling", "book": "Advances in Neural Information Processing Systems", "page_first": 995, "page_last": 1004, "abstract": null, "full_text": "Multi-effect Decompositions \nfor Financial Data Modeling \n\nLizhong Wu &  John Moody \n\nOregon Graduate Institute, Computer Science Dept., \n\nPO Box 91000, Portland, OR 97291 \n\nalso at: \n\nNonlinear Prediction Systems, \n\nPO Box 681, University Station, Portland, OR 97207 \n\nAbstract \n\nHigh  frequency  foreign  exchange  data  can  be  decomposed  into  three \ncomponents:  the  inventory effect component,  the  surprise infonnation \n(news) component and the regular infonnation component. The presence \nof the inventory effect and news can make analysis of trends due to  the \ndiffusion of infonnation (regular information component) difficult. \nWe propose a neural-net-based, independent component analysis to sep(cid:173)\narate high frequency foreign exchange data into these three components. \nOur empirical results show that our proposed multi-effect decomposition \ncan reveal the intrinsic price behavior. \n\n1 \n\nIntroduction \n\nTick-by-tick, high frequency foreign exchange rates are extremely noisy and volatile, but \nthey are not simply pure random walks (Moody & Wu  1996).  The price movements are \ncharacterized by a  number of \"stylized facts\" \n,including the  following  two  properties: \n(1) short tenn, weak oscillations on a time scale of several ticks and (2) erratic occurrence \nof turbulence lasting from minutes to tens of minutes.  Property (1) is  most likely caused \nby the market makers' inventory effect (O'Hara 1995), and property (2) is due to surprise \ninformation, such as news, rumors, or major economic announcements. The price changes \ndue to property (1) are referred to as the inventory effect component, and the changes due \nto property (2)  are referred to as the surprise infonnation component.  The price changes \ndue to other infonnation is referred to as the regular infonnation component. \n\nIThis terminology is borrowed from the financial economics literature.  For additional properties \nof high frequency foreign exchange price series. see (Guilaumet, Dacorogna. Dave. Muller. Olsen & \nPictet 1994). \n\n\f996 \n\nL.  Wu and 1.  E.  Moody \n\nDue to the inventory effect, price changes show strong negative correlations on short time \nscales (Moody &  Wu  1995).  Because of the surprise infonnation effect, distributions  of \nprice  changes are  non-nonnal  (Mandelbrot  1963).  Since  both  the  inventory  effect  and \nthe  surprise  information  effect are  short  tenn and  temporary,  their  corresponding price \ncomponents are independent of the  fundamental price changes.  However, their existence \nwill  seriously affect data analysis and modeling (Moody &  Wu  1995).  Furthennore,  the \nmost reliable component of price changes, for forecasting purposes, is the long tenn trend. \nThe presence of high frequency oscillations and short periods ofturbulence make it difficult \nto identify and predict the changes in such trends, if they occur. \n\nIn this paper, we propose a novel approach with the following price model: \n\nq(t) = CIPI(t) + C2P2(t) + C3P3(t) + e(t). \n\n(1) \nIn  this  model,  q(t)  is  the  observed  price  series  and PI (t),  P2(t)  and P3(t)  correspond \nrespectively to the regular infonnation component, the surprise infonnation component and \nthe inventory effect component.  PI (t), P2 (t)  and P3 (t) are mutually independent and may \nindividually be either iid  or correlated.  e(t)  is  process noise, and c},  C2  and C3  are scale \nconstants.  Our goal is to find PI (t), p2(t) and P3( t)  given q(t). \nThe outline of the paper is as follows.  We describe our approach for multi-effect decompo(cid:173)\nsition in Section 2.  In Section 3, we analyze the decomposed price components obtained \nfor the high frequency foreign exchange rates and characterize their stochastic properties. \nWe  conclude and discuss  the  potential  applications of our multi-effect decomposition in \nSection 4. \n\n2  Multi-effect Decomposition \n\n2.1 \n\nIndependent Source Separation \n\nThe task of decomposing the observed price quotes into a regular infonnation component, \na surprise infonnation component and an inventory effect component can be exactly fitted \ninto the framework of independent source separation.  Independent source separation can \nbe described as follows: \n\nAssume  that  X  = {Xi, i  = 1, 2, ... , n}  are  the  sensor outputs  which \nare  some superposition of unknown independent sources S  =  {Si' i  = \n1, 2, ... , m }.  The  task  of independent  source  separation  is  to  find  a \nmapping Y  =  f (X), so that Y  ~ AS, where A is an m  x  m  matrix in \nwhich each row and column contains only one non-zero element. \n\nApproaches to separate statistically-independent components in the inputs include \n\n\u2022  Blind source separation (Jutten &  Herault 1991), \n\u2022  Infonnation maximization (Linsker 1989), (Bell &  Sejnowski 1995), \n\u2022  Independent component analysis, (Comon 1994), (Amari, Cichocki & Yang 1996), \n\u2022  Factorial coding (Barlow 1961). \n\nAll  of these approaches can be  implemented by artificial  neural  networks.  The network \narchitectures can be linear or nonlinear, multi-layer perceptrons, recurrent networks or other \ncontext sensitive networks (pearlmutter &  Parra 1997).  We can choose a training criterion \nto  minimize  the  energy  in  the  output  units,  to  maximize  the  infonnation  transferred  in \nthe  network,  to  reduce  the  redundancies  between  the  outputs,  or to  use  the  Edgeworth \nexpansion  or  Gram-Charlier expansion  of a  probability  distribution,  which  leads  to  an \nanalytic expression of the entropy in tenns of measurable higher order cumulants. \n\n\fMulti-effect Decompositions/or Financial Data Modeling \n\nr1(t) \n\nOrthogonal \n\nq(t) \n\nMulti-scale \n\nSmoothing  e  r2(t) \n\nIndependent \n\nComponent \n\nDecomposition \n\nr3(t) \n\nAnalysis \n\n997 \n\n~(t) \n\n~(t) \n\nFigure  1:  System diagram of multi-effect decomposition for  high frequency  foreign  ex(cid:173)\nchange rates.  q(t) are original price quotes, ri(t) are the reference inputs, and Pie t) are the \ndecomposed components. \n\nFor our price decomposition problem, the non-Gaussian nature of price series requires that \nthe transfer function of the decomposition system be nonlinear.  In general, the nonlinearities \nin the transfer function are able to pick up higher order moments of the input distributions \nand perfonn higher order statistical redundancy reduction between outputs. \n\n2.2  Reference input selection \n\nIn traditional approaches to blind source separation, nothing is assumed to be known about \nthe inputs, and the systems adapt on-line and without a supervisor.  This works only if the \nnumber of sensors is  not less  than the  number of independent sources.  If the number of \nsensors is  less than that of sources, the  sources can, in  theory,  be separated into disjoint \ngroups (Cao & Liu  1996).  However, the problem is ill-conditioned for most of the above \npractical approaches which only consider the case where the number of sensors is equal to \nthe number of sources. \n\nIn  our task to decompose the  multiple  components of price  quotes,  the  problem  can be \ndivided  into  two  cases.  If the  prices are  sampled at regular intervals,  we can use price \nquotes  observed  in  different  markets,  and  have  the  number of sensors  be equal  to  the \nnumber of price  components.  However, in the  high frequency markets,  the price quotes \nare not regularly spaced in time. Price quotes from different markets will not appear at the \nsame time,  so we cannot apply the  price quotes from different markets  to  the system.  In \nthis case, other reference inputs are needed. \n\nMotivated by the  use of reference inputs  for noise canceling (Widrow,  Glover,  McCool, \nKaunitz,  Williams,  Heam, Zeidler,  Dong & Goodlin  1975), we generate three reference \ninputs from original price quotes.  They are the estimates of the three desired components. \nIn the following, we briefly describe our procedure for generating the reference inputs. \nBy modeling the price quotes using a \"True Price\" state space model (Moody & Wu 1996) \n\nq(t) =  rl (t) + r3(t)  , \n\n(2) \n\nwhere rl (t) is an estimate of the infonnation component (True Price) and r3( t) is an estimate \nof the  inventory effect component (additive  noise),  and by assuming that the  True  Price \nrl (t)  is  a  fractional  Brownian  motion  (Mandelbrot &  Van  Ness  1968),  we  can estimate \nrl (t)  and r3 (t)  with given q( t), (Moody & Wu  1996), as \n\nm,n \n\n(3) \n\n(4) \n\n\f998 \n\nL.  Wu and J.  E.  Moody \n\nFigure  2:  Multi-effect decompositions for  two  segments  of the  DEMIUSD  (log  prices) \nextracted from  September 1995.  The three  panels in each segment display the observed \nprices (the dotted curve in  upper panel), the  regular information component (solid curve \nin upper panel), the surprise information component (mid panel) and the inventory effect \ncomponent (lower panel). \n\nwhere  t/!::'(t)  is  an  orthogonal  wavelet  function,  Q::'  is  the  coefficient  of the  wavelet \ntransform of q(t), m is the index of the scales and n is the time index of the components in \nthe wavelet transfer, S(m,B) is a smoothing function, and its parameters can be estimated \nusing the EM algorithm (Womell &  Oppenheim 1992). \n\nWe then estimate the surprise information component as the residual between the informa(cid:173)\ntion component and its moving average: \n\nr2(t)  =  r}(t) - set) \n\n(5) \n\nset) is an exponential moving average of rl(t) and \n\nset) = (1 + a)rl(t) - as(t - 1) \n\n(6) \nwhere  a  is  a  factor.  Although  it  can  be  optimized  based on  the  training  data,  we  set \na = -0.9 in our current work. \nOur system diagram for  multi-effect decomposition is  shown in  Figure  1.  Using  multi(cid:173)\nscale decomposition Eqn(3) and smoothing techniques Eqn(6),  we  obtain three reference \ninputs.  We  can then separate the reference inputs into three independent components via \nindependent component analysis using an artificial neural network.  Figure 2 presents multi(cid:173)\neffect decompositions for two segments of the DEMIUSD rates.  The first segment contains \nsome impulses,  and  the  corresponding  surprise  information  component is  able  to  catch \nsuch volatile  movements.  The second segment is basically down trending, so its  surprise \ninformation component is comparatively flat. \n\n3  Empirical Analysis \n\n3.1  Mutually Independent Analysis \n\nMutual independence of the  variables is  satisfied if the joint probability density function \nequals  the  product of the  marginal densities,  or equivalently,  the  characteristic  function \nsplits into the sum of marginal characteristic functions:  g(X) =  E7=1 gi(Xi).  Taking the \nTaylor expansion of both sides of the above equation, products between different variables \nXi  in  the left-hand side must be zero since there are no  such terms in the right-hand side. \n\n\fMulti-effect Decompositions for Financial Data Modeling \n\n999 \n\nTable  1:  Comparisons between the  correlation coefficients p  (nonnalized) and the cross(cid:173)\ncumulants r (unnonnalized) of order 4 before and after independent component analysis \n(lCA).  The DEMIUSD quotes  for  September 1995 is divided  into  147  sub-sets of 1024 \nticks.  The results  presented here are  the  median values.  The last column is  the  absolute \nratio of before ICA and after ICA. We note that all ratios are greater than I, indicating that \nafter ICA, the components become more independent. \n\nComponents \n\npairs \n\nCross-\n\nCumulants \n\nPl(t) \"\" P2(t) \n\npl(t) \"\" P3(t) \n\nP2(t)  \"\" P3(t) \n\nP12 \nr13 \nr 22 \nr31 \nP13 \nr13 \nr22 \nr31 \nP23 \nr13 \nr22 \nr 31 \n\nBefore \nICA \n0.56 \n2.7e-14 \n-5.6e-15 \n2.0e-11 \n0.15 \n2.le-15 \n-2.0e-15 \n5.ge-12 \n0.17 \n9.le-16 \n1.2e-15 \n3.6e-15 \n\nAfter \nICA \n0.14 \n7.8e-17 \n9.2e-16 \n1.3e-13 \n0.03 \n1.6e-17 \n-4.5e-16 \n6.ge-14 \n0.04 \n-5.0e-19 \n4.ge-17 \n3.0e-17 \n\nAbsolute \nratio \n4.1 \n342.2 \n6.0 \n148.5 \n4.7 \n128.9 \n4.5 \n84.5 \n4.3 \n1806.0 \n24.3 \n119.6 \n\nWe observe the cross-cumulants of order 4: \n\nM13  - 3M2oMll \n\nr13 \nr 22  =  M22  - M20M02  - 2M?1 \nr 31  =  M31  - 3M02Mll \n\n(7) \n(8) \n(9) \nwhere M lei  =  E { x~ x~} denote the moments of order k + I.  If x i  and x j  are independent, \nthen  their  cross-cumulants  must  be  zero  (Comon  1994).  Table  I  compares  the  cross(cid:173)\ncumulants before and after independent component analysis (ICA) for the  DEMIUSD in \nSeptember 1995.  For reference, the correlation coefficients before and after ICA are also \nlisted in the table.  We see that after ICA, the components have become less correlated and \nthus more independent. \n\n3.2  Autocorrelation Analysis \n\nFigure  3  depicts  the  autocorrelation  functions  of the  changes in  individual  components \nand compares them to the original returns.  We compute the short-run autocorrelations for \nthe lags up to 50.  Figure 3 gives the means and standard deviations for September 1995. \nFrom  the  figure,  we  can see that both  the  inventory effect  component and  the  original \nreturns show very similar autocorrelation functions, which are dominated by the significant \nnegative, first-order  autocorrelations.  The mean values for the other orders are basically \nequal to zero.  The autocorrelations of the regular information component and the surprise \ninformation component show positive  correlations except at first  order.  These non-zero \nautocorrelations are  hidden by noise in the original  series.  The autocorrelation function \nof the  surprise  information component decays faster than  that of the  regular information \ncomponent. On average, it is below the 95% confidence band for lags larger than 20 ticks. \n\nThe above autocorrelation analysis suggests the  following.  (I) Price changes due to the \ninformation effects are slightly trending on tick-by-tick time scales. The trend in the surprise \ninformation component is shorter term than that in the regular information component. \n\n\f1000 \n\nL.  Wu and J. E.  Moody \n\n0.6 \n\n0.4 \na  0.2 \n~  0  mm.mmmmHHim!lH!ml!l)mtmm \n\u00ab \n\n-0.2 \n\n-0.4 \n\n0.6 \n\n0.4 \n\n~ 0.2 \n~  0 \n\" \u00ab -0.2 \n-0.4 \n\n0.6 \n\n0.4 \n\n~ 0.2 \n~  0 \n\" \u00ab -0.2 \n-0.4 \n\n0.6 \n\n0.4 \n\n~ 0.2 \n~  0 \n\" \u00ab -0.2 \n-0.4 \n\n0 \n\n10 \n\n20 \n\n30 \n\n40 \n\n0 \n\n10 \n\n20 \n\n30 \n\n40 \n\nFigure 3:  Comparison of autocorrelation functions of the changes in the original observed \nprices (the upper-left panel), the inventory effect component (the lower-left panel), the regu(cid:173)\nlar information component (the upper-right panel) and the surprise infonnation component \n(the lower-right panel).  The results presented are means and standard deviations, and the \nhorizontal dotted lines represent the 95% confidence band.  The DEMIUSD in September \n1995 is divided into 293 sub-sets of 1024 ticks with overlapping of 512 ticks. \n\n(2) The autocorrelation function  of original returns reflects only the price changes due to \nthe  inventory effect.  This  further confirms  that  the  existence of short tenn memory  can \nmislead the  analysis of dependence on longer time  scales.  Subsequently, we  can  see the \nusefulness of the  multi-effect decomposition.  Our empirical results  should be viewed as \npreliminary,  since  they  may  depend  upon  the  choice  of True  Price  model.  Additional \nstudies are ongoing. \n\n4  Conclusion and Discussion \n\nWe  have  developed  a  neural-net-based  independent component  analysis  (ICA)  for  the \nmulti-effect decomposition of high frequency financial data.  Empirical results with foreign \nexchange rates have demonstrated that the decomposed components are mutually indepen(cid:173)\ndent.  The obtained regular infonnation component has recovered the trending behavior of \nthe intrinsic price movements. \n\nPotential applications for multi-effect decompositions include: \n(1)  outlier detection and filtering:  Filtering  techniques for removing various noisy effects \nand identifYing long tenn trends have been widely studied (see for example Assimakopou(cid:173)\nlos  (1995\u00bb.  MuIti-effect  decompositions  provide  us  with  an  alternative  approach.  As \ndemonstrated in Section 3,  the  regular infonnation  component can,  in  most cases, catch \nrelatively stable and longer tenn trends originally embedded in the price quotes. \n(2)  devolatilization:  Price  series  are  heteroscedastic  (Boilers lev,  Chou  &  Kroner  1992). \nDevolatilization  has  been  widely  studied (see,  for example,  Zhou  (1995\u00bb.  The  regular \ninfonnation component obtained from our multi-effect decomposition appears less volatile, \nand furthennore, its volatility changes more smoothly compared to the original prices. \n(3) mixture of local experts modeling:  In most cases, one might be interested in only stable, \nlong tenn trends of price movements.  However, the surprise infonnation and inventory ef(cid:173)\nfect components are not totally useless.  By decomposing the price series into three mutually \n\n\fMulti-effect Decompositionsfor Financial Data Modeling \n\n1001 \n\nindependent components, the prices can be modeled by a mixture of local experts (Jacobs, \nJordan &  Barto 1990), and better modeling perfonnances can be expected. \n\nReferences \nAmari, S., Cichocki, A.  & Yang.  H.  (1996), A  new learning algorithm  for blind signal separation, \nin D. Touretzky. M.  Mozer & M.  Hasselmo, eds.  'Advances in Neural Infonnation Processing \nSystems 8', MIT Press:  Cambridge, MA. \n\nAssimakopoulos. V.  (1995), 'A successive filtering technique for identifying long-tenn trends'. Jour(cid:173)\n\nnal of Forecasting 14, 35-43. \n\nBarlow,  H.  (1961),  Possible  principles  underlying  the  transfonnation  of  sensory  messages,  in \n\nW.  Rosenblith, ed., 'Sensory Communication', MIT Press:  Cambridge, MA, pp. 217-234. \n\nBell,  A.  &  Sejnowski, T.  (1995),  'An infonnation-maximization approach  to blind separation and \n\nblind deconvolution', Neural Computation 7(6), 1129-1159. \n\nBollerslev, T.,  Chou, R.  &  Kroner. K.  (1992).  'ARCH modelling in finance:  A review of the theory \n\nand empirical evidence', Journal of Econometrics 8, 5-59. \n\nCao, X. & Liu, R. (1996), 'General approach to blind source separation' \u2022 IEEE Transactions on Signal \n\nProcessing 44(3), 562-569. \n\nComon, P.  (1994), 'Independent component analysis, a new concept?', Signal Process 36,287-314. \nGuilaumet, D., Dacorogna, M., Dave, R., Muller, U., Olsen, R.  & Pictet, O.  (1994), From the bird's \neye to the microscope, a survey of new stylized facts of the intra-daily foreign exchange markets, \nTechnical Report DMG.1994-04-06, Olsen & Associates, Zurich, Switzerland. \n\nJacobs.  R.,  Jordan,  M.  & Barto, A.  (1990), Task decomposition through competition in a  modular \nconnectionist architecture:  The what and where vision tasks, Technical Report COINS 90-27, \nDepartment of Brain & Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, \nMA. \n\nJutten, C. &  Herault, 1.  (1991).  'Blind separation of sources. part I: An adaptive algorithm based on \n\nneuromimetic architecture' , Signal Process 24( 1), 1-10. \n\nLinsker,  R.  (1989), An application of the  principle of maximum infonnation preservation to linear \nsystems, in D. Touretzky, ed., 'Advances in Neural Infonnation Processing Systems 1', Morgan \nKaufmann Publishers, San Francisco, CA. \n\nMandelbrot, B. (1963), 'The variation of certain specUlative prices' , Journal of Business 36, 394--419. \nMandelbrot, B.  &  Van  Ness, J.  (1968),  'Fractional Brownian motion, fractional noise.  and applica(cid:173)\n\ntions', SIAM Review 10. \n\nMoOdy, 1.  & Wu. L.  (1995), Statistical analysis and forecasting of high frequency foreign exchange \nrates,  in  'The  First  International  Conference  on  High  Frequency  Data  in  Finance',  Zurich, \nSwitzerland. \n\nMoody, J. & Wu, L. (1996), What is the 'True Price '? - state space models for high frequency financial \ndata, in  S.  Amari, L.  Xu, L. Chan, I.  King & K.  Leung, eds,  'Progress in Neural Infonnation \nProcessing (Proceedings of ICONIPS*96, Hong Kong)', Springer-Verlag, Singapore. pp. 697-\n704, Vo1.2. \n\nO'Hara, M. (1995), Market Microstructure Theory, Blackwell Business. \nPearl mutter,  B.  A.  &  Parra,  L.  (1997),  Maximum  likelihood  blind  source  separation:  a  context(cid:173)\nsensitive generalization ofICA, in M. Mozer, M. Jordan & T. Petsche, eds, 'Advances in Neural \nInfonnation Processing Systems 9' , MIT Press:  Cambridge, MA. \n\nWidrow,  B.,  Glover,  J.,  McCool, 1.,  Kaunitz,  J.,  Williams,  C.,  Hearn,  R.,  Zeidler,  J.,  Dong,  E.  & \nGoodlin,  R.  (1975),  'Adaptive  noise  cancelling:  principles and  applications',  Proceedings of \nIEEE 63(12), 1692-1716. \n\nWornell, G. &  Oppenheim, A. (1992),  'Estimation of fractal signals from noisy measurements using \n\nwavelets', IEEE Transactions on Signal Processing 40(3), 611-623. \n\nZhou, B. (1995), Forecasting foreign exchange rates series subject to de-volatilization, in  'The First \n\nInternational Conference on High Frequency Data in Finance', Zurich, Switzerland. \n\n\f\fPART IX \n\nCONTROL, NAVIGATION AND  PLANNING \n\n\f\f", "award": [], "sourceid": 1251, "authors": [{"given_name": "Lizhong", "family_name": "Wu", "institution": null}, {"given_name": "John", "family_name": "Moody", "institution": null}]}