{"title": "Inferring State Sequences for Non-linear Systems with Embedded Hidden Markov Models", "book": "Advances in Neural Information Processing Systems", "page_first": 401, "page_last": 408, "abstract": "", "full_text": "Inferring State Sequences for Non-linear\n\nSystems with Embedded Hidden Markov Models\n\nRadford M. Neal, Matthew J. Beal, and Sam T. Roweis\n\nDepartment of Computer Science\n\nUniversity of Toronto\n\nToronto, Ontario, Canada M5S 3G3\n\nfradford,beal,roweisg@cs.utoronto.ca\n\nAbstract\n\nWe describe a Markov chain method for sampling from the distribution\nof the hidden state sequence in a non-linear dynamical system, given a\nsequence of observations. This method updates all states in the sequence\nsimultaneously using an embedded Hidden Markov Model (HMM). An\nupdate begins with the creation of \u201cpools\u201d of candidate states at each\ntime. We then de\ufb01ne an embedded HMM whose states are indexes within\nthese pools. Using a forward-backward dynamic programming algo-\nrithm, we can ef\ufb01ciently choose a state sequence with the appropriate\nprobabilities from the exponentially large number of state sequences that\npass through states in these pools. We illustrate the method in a simple\none-dimensional example, and in an example showing how an embed-\nded HMM can be used to in effect discretize the state space without any\ndiscretization error. We also compare the embedded HMM to a particle\nsmoother on a more substantial problem of inferring human motion from\n2D traces of markers.\n\n1\n\nIntroduction\n\nConsider a dynamical model in which a sequence of hidden states, x = (x0; : : : ; xn(cid:0)1), is\ngenerated according to some stochastic transition model. We observe y = (y0; : : : ; yn(cid:0)1),\nwith each yt being generated from the corresponding xt according to some stochastic ob-\nservation process. Both the xt and the yt could be multidimensional. We wish to randomly\nsample hidden state sequences from the conditional distribution for the state sequence given\nthe observations, which we can then use to make Monte Carlo inferences about this poste-\nrior distribution for the state sequence. We suppose in this paper that we know the dynamics\nof hidden states and the observation process, but if these aspects of the model are unknown,\nthe method we describe will be useful as part of a maximum likelihood learning algorithm\nsuch as EM, or a Bayesian learning algorithm using Markov chain Monte Carlo.\n\nIf the state space is \ufb01nite, of size K, so that this is a Hidden Markov Model (HMM), a\nhidden state sequence can be sampled by a forward-backwards dynamic programming al-\ngorithm in time proportional to nK 2 (see  for a review of this and related algorithms).\nIf the state space is