Dynamical systems evolve according to the repeated application of a map from a state space to itself, possibly in the presence of noise. The two most common types of dynamical systems are topological systems, which consist of a continuous function from a compact set to itself, and stationary systems, which consist of a function from a set (measurable space) to itself that preserves a probability measure of interest. Topological systems have been applied to the study of physical phenomena arising in a variety of fields, ranging from ecology and systems biology to chemistry. Stationary systems describe stationary stochastic processes, and are fundamental to the modeling of time series as well as physical phenomena in their steady state. A stationary system is ergodic if it cannot be decomposed into stationary subsystems.
Statistical inference from ergodic systems has been a long standing interest of our group. In recent work we have established that a variety of inference problems involving dynamical systems and ergodic processes can be studied and analyzed in terms of a single family of problems that we call optimal tracking. In an optimal tracking problem, we wish to match the initial trajectories of an observed ergodic system with trajectories of a reference topological system. Specifically, we choose an initial state of the reference system yielding a trajectory with minimum average cost relative to the observed trajectory. One can show that the average cost of the optimal matching converges, and that the limit has a variational characterization in terms of joinings (dynamically invariant couplings) of the observed and reference systems. Moreover, the set of optimal joinings (those yielding the limiting average cost) is convex and compact.
It turns out that there is a formal connection between the tracking problem and a variety of statistical estimation schemes based on empirical risk minimization. Using this connection, we have derived a general convergence theorem for empirical risk minimization based inference procedures that addresses the important issues of model misspecification and identifiability in a direct and natural way. As one application of the theorem, we recover classical results on the consistency of maximum likelihood estimation. Another application of the theorem yields new results on the identification of dynamical systems from quantized observations.
In related work, we consider the problem of fitting a parametric family of topological systems to an observed ergodic process using empirical risk minimization. We show that the empirically optimal parameters converge to a limiting set that is characterized through a projection of the observed process onto a set of processes associated with systems in the family. In the case where the observed system is an ergodic process plus i.i.d. noise, we show that the fitting process is unaffected by the noise if the family of model systems has limited complexity. Complexity is quantified by a notion of entropy for families of infinite sequences that has close connections with topological entropy and mean widths for stationary processes.