Tighter bounds on the expressivity of transformer encoders

D Chiang, P Cholak, A Pillay - International Conference on …, 2023 - proceedings.mlr.press
Characterizing neural networks in terms of better-understood formal systems has the
potential to yield new insights into the power and limitations of these networks. Doing so for …

[图书][B] Elements of finite model theory

L Libkin - 2004 - Springer
Finite model theory is an area of mathematical logic that grew out of computer science
applications. The main sources of motivational examples for finite model theory are found in …

An optimal lower bound on the number of variables for graph identification

JY Cai, M Fürer, N Immerman - Combinatorica, 1992 - Springer
In this paper we show that Ω (n) variables are needed for first-order logic with counting to
identify graphs on n vertices. The k-variable language with counting is equivalent to the (k …

[图书][B] Descriptive complexity

N Immerman - 1998 - books.google.com
A basic issue in computer science is the complexity of problems. Computational complexity
measures how much time or memory is needed as a function of the input problem size …

[图书][B] Algorithms and theory of computation handbook, volume 2: special topics and techniques

MJ Atallah, M Blanton - 2009 - books.google.com
This handbook provides an up-to-date compendium of fundamental computer science
topics, techniques, and applications. Along with updating and revising many of the existing …

What formal languages can transformers express? a survey

L Strobl, W Merrill, G Weiss, D Chiang… - Transactions of the …, 2024 - direct.mit.edu
As transformers have gained prominence in natural language processing, some researchers
have investigated theoretically what problems they can and cannot solve, by treating …

[图书][B] Describing graphs: A first-order approach to graph canonization

N Immerman, E Lander - 1990 - Springer
In this paper we ask the question,“What must be added to first-order logic plus least-fixed
point to obtain exactly the polynomial-time properties of unordered graphs?” We consider …

A catalog of complexity classes

DS Johnson - Algorithms and complexity, 1990 - Elsevier
Publisher Summary This chapter discusses the concepts needed for defining the complexity
classes. A complexity class is a set of problems of related resource-based complexity. A …

On complexity and optimization of expensive queries in complex event processing

H Zhang, Y Diao, N Immerman - Proceedings of the 2014 ACM SIGMOD …, 2014 - dl.acm.org
Pattern queries are widely used in complex event processing (CEP) systems. Existing
pattern matching techniques, however, can provide only limited performance for expensive …

A logic for expressing log-precision transformers

W Merrill, A Sabharwal - Advances in Neural Information …, 2024 - proceedings.neurips.cc
One way to interpret the reasoning power of transformer-based language models is to
describe the types of logical rules they can resolve over some input text. Recently, Chiang et …