Transport in reservoir computing

Abstract

Reservoir computing systems are essentially dynamical systems influenced by an exogenous input. Such systems are extensively used in biologically inspired information processing, and are the state-of-the art techniques for several machine learning tasks. If the statistics of the response or output of the system depends discontinuously on the distribution of the inputs, a fundamental challenge arises in applications where inherent changes in the input stochastic source or noise are expected. This problem can be experimentally demonstrated by showing that altering input statistics can drastically affect the statistics of the response. We solve this instability problem by providing sufficient conditions under which both the marginals and the joint distributions of the response depend continuously on that of the input. To prove our results, we establish the existence of an invariant measure and show that its dependence on the input process is continuous when the processes are endowed with the Wasserstein distance. The main tool in these developments is the characterization of those invariant measures as fixed points of naturally defined Foias operators that appear in this context and which are examined extensively in the paper. These fixed points are obtained by imposing a newly introduced stochastic state contractivity on the driven system that is readily verifiable in examples. Stochastic state contractivity can be satisfied by systems that are not state-contractive, which is a need typically evoked to guarantee the echo state property in reservoir computing. As a result, it may actually be satisfied even if the echo state property is missing.