Design of Experiments via Information Theory

Part of Advances in Neural Information Processing Systems 16 (NIPS 2003)

Bibtex Metadata Paper

Authors

Liam Paninski

Abstract

We discuss an idea for collecting data in a relatively efficient manner. Our point of view is Bayesian and information-theoretic: on any given trial, we want to adaptively choose the input in such a way that the mutual in- formation between the (unknown) state of the system and the (stochastic) output is maximal, given any prior information (including data collected on any previous trials). We prove a theorem that quantifies the effective- ness of this strategy and give a few illustrative examples comparing the performance of this adaptive technique to that of the more usual nonadap- tive experimental design. For example, we are able to explicitly calculate the asymptotic relative efficiency of the “staircase method” widely em- ployed in psychophysics research, and to demonstrate the dependence of this efficiency on the form of the “psychometric function” underlying the output responses.