Formal and functional theories of language seem disparate, because formal theories answer
the question of what a language is, while functional theories answer the question of what …

This work focuses on explaining both grammatical universals of word order and quantitative
word-order preferences in usage by means of a simple efficiency principle: dependency …

Memory limitations are known to constrain language comprehension and production, and
have been argued to account for crosslinguistic word order regularities. However, a …

A stochastic process' statistical complexity stands out as a fundamental property: the
minimum information required to synchronize one process generator to another. How much …

Even simply defined, finite-state generators produce stochastic processes that require
tracking an uncountable infinity of probabilistic features for optimal prediction. For processes …

Predictive rate-distortion analysis suffers from the curse of dimensionality: clustering
arbitrarily long pasts to retain information about arbitrarily long futures requires resources …

As we discuss, a stationary stochastic process is nonergodic when a random persistent topic
can be detected in the infinite random text sampled from the process, whereas we call the …

We present two examples of finite-alphabet, infinite excess entropy processes generated by
stationary hidden Markov models (HMMs) with countable state sets. The first, simpler …

Written language is complex. A written text can be considered an attempt to convey a
meaningful message which ends up being constrained by language rules, context …

Complexity is a hallmark of intelligent behavior consisting both of regular patterns and
random variation. To quantitatively assess the complexity and randomness of human …