We introduce an ambidextrous view of stochastic dynamical systems, comparing their
forward-time and reverse-time representations and then integrating them into a single time …

It was observed that large language models exhibit a power-law decay of cross entropy with
respect to the number of parameters and training tokens. When extrapolated literally, this …

As we discuss, a stationary stochastic process is nonergodic when a random persistent topic
can be detected in the infinite random text sampled from the process, whereas we call the …

This paper aims to bring theoretical linguistics and cognition-general theories of learning
into closer contact. I argue that linguists' notions of rich Universal Grammars (UGs) are well …

We present a hypothetical argument against finite-state processes in statistical language
modeling that is based on semantics rather than syntax. In this theoretical model, we …

We review recent progress in understanding the meaning of mutual information in natural
language. Let us define words in a text as strings that occur sufficiently often. In a few …

Constant entropy rate (conditional entropies must remain constant as the sequence length
increases) and uniform information density (conditional probabilities must remain constant …

We give a systematic expansion of the crypticity—a recently introduced measure of the
inaccessibility of a stationary process's internal state information. This leads to a hierarchy of …

Biomolecular communication demands that interactions between parts of a molecular
system act as scaffolds for message transmission. It also requires an evolving and organized …

Suppose that we have a method which estimates the conditional probabilities of some
unknown stochastic source and we use it to guess which of the outcomes will happen. We …