-
On dimensionality of feature vectors in MPNNs
Authors:
César Bravo,
Alexander Kozachinskiy,
Cristóbal Rojas
Abstract:
We revisit the classical result of Morris et al.~(AAAI'19) that message-passing graphs neural networks (MPNNs) are equal in their distinguishing power to the Weisfeiler--Leman (WL) isomorphism test.
Morris et al.~show their simulation result with ReLU activation function and $O(n)$-dimensional feature vectors, where $n$ is the number of nodes of the graph. By introducing randomness into the arch…
▽ More
We revisit the classical result of Morris et al.~(AAAI'19) that message-passing graphs neural networks (MPNNs) are equal in their distinguishing power to the Weisfeiler--Leman (WL) isomorphism test.
Morris et al.~show their simulation result with ReLU activation function and $O(n)$-dimensional feature vectors, where $n$ is the number of nodes of the graph. By introducing randomness into the architecture, Aamand et al.~(NeurIPS'22) were able to improve this bound to $O(\log n)$-dimensional feature vectors, again for ReLU activation, although at the expense of guaranteeing perfect simulation only with high probability.
Recently, Amir et al.~(NeurIPS'23) have shown that for any non-polynomial analytic activation function, it is enough to use just 1-dimensional feature vectors. In this paper, we give a simple proof of the result of Amit et al.~and provide an independent experimental validation of it.
△ Less
Submitted 14 February, 2024; v1 submitted 6 February, 2024;
originally announced February 2024.
-
Towards Simpler Sorting Networks and Monotone Circuits for Majority
Authors:
Natalia Dobrokhotova-Maikova,
Alexander Kozachinskiy,
Vladimir Podolskii
Abstract:
In this paper, we study the problem of computing the majority function by low-depth monotone circuits and a related problem of constructing low-depth sorting networks. We consider both the classical setting with elementary operations of arity $2$ and the generalized setting with operations of arity $k$, where $k$ is a parameter. For both problems and both settings, there are various constructions…
▽ More
In this paper, we study the problem of computing the majority function by low-depth monotone circuits and a related problem of constructing low-depth sorting networks. We consider both the classical setting with elementary operations of arity $2$ and the generalized setting with operations of arity $k$, where $k$ is a parameter. For both problems and both settings, there are various constructions known, the minimal known depth being logarithmic. However, there is currently no known construction that simultaneously achieves sub-log-squared depth, effective constructability, simplicity, and has a potential to be used in practice. In this paper we make progress towards resolution of this problem.
For computing majority by standard monotone circuits (gates of arity 2) we provide an explicit monotone circuit of depth $O(\log_2^{5/3} n)$. The construction is a combination of several known and not too complicated ideas.
For arbitrary arity of gates $k$ we provide a new sorting network architecture inspired by representation of inputs as a high-dimensional cube. As a result we provide a simple construction that improves previous upper bound of $4 \log_k^2 n$ to $2 \log_k^2 n$. We prove the similar bound for the depth of the circuit computing majority of $n$ bits consisting of gates computing majority of $k$ bits. Note, that for both problems there is an explicit construction of depth $O(\log_k n)$ known, but the construction is complicated and the constant hidden in $O$-notation is huge.
△ Less
Submitted 18 October, 2023;
originally announced October 2023.
-
Logical Languages Accepted by Transformer Encoders with Hard Attention
Authors:
Pablo Barcelo,
Alexander Kozachinskiy,
Anthony Widjaja Lin,
Vladimir Podolskii
Abstract:
We contribute to the study of formal languages that can be recognized by transformer encoders. We focus on two self-attention mechanisms: (1) UHAT (Unique Hard Attention Transformers) and (2) AHAT (Average Hard Attention Transformers). UHAT encoders are known to recognize only languages inside the circuit complexity class ${\sf AC}^0$, i.e., accepted by a family of poly-sized and depth-bounded boo…
▽ More
We contribute to the study of formal languages that can be recognized by transformer encoders. We focus on two self-attention mechanisms: (1) UHAT (Unique Hard Attention Transformers) and (2) AHAT (Average Hard Attention Transformers). UHAT encoders are known to recognize only languages inside the circuit complexity class ${\sf AC}^0$, i.e., accepted by a family of poly-sized and depth-bounded boolean circuits with unbounded fan-ins. On the other hand, AHAT encoders can recognize languages outside ${\sf AC}^0$), but their expressive power still lies within the bigger circuit complexity class ${\sf TC}^0$, i.e., ${\sf AC}^0$-circuits extended by majority gates. We first show a negative result that there is an ${\sf AC}^0$-language that cannot be recognized by an UHAT encoder. On the positive side, we show that UHAT encoders can recognize a rich fragment of ${\sf AC}^0$-languages, namely, all languages definable in first-order logic with arbitrary unary numerical predicates. This logic, includes, for example, all regular languages from ${\sf AC}^0$. We then show that AHAT encoders can recognize all languages of our logic even when we enrich it with counting terms. We apply these results to derive new results on the expressive power of UHAT and AHAT up to permutation of letters (a.k.a. Parikh images).
△ Less
Submitted 5 October, 2023;
originally announced October 2023.
-
Simple online learning with consistent oracle
Authors:
Alexander Kozachinskiy,
Tomasz Steifer
Abstract:
We consider online learning in the model where a learning algorithm can access the class only via the \emph{consistent oracle} -- an oracle, that, at any moment, can give a function from the class that agrees with all examples seen so far. This model was recently considered by Assos et al.~(COLT'23). It is motivated by the fact that standard methods of online learning rely on computing the Littles…
▽ More
We consider online learning in the model where a learning algorithm can access the class only via the \emph{consistent oracle} -- an oracle, that, at any moment, can give a function from the class that agrees with all examples seen so far. This model was recently considered by Assos et al.~(COLT'23). It is motivated by the fact that standard methods of online learning rely on computing the Littlestone dimension of subclasses, a computationally intractable problem.
Assos et al.~gave an online learning algorithm in this model that makes at most $C^d$ mistakes on classes of Littlestone dimension $d$, for some absolute unspecified constant $C > 0$. We give a novel algorithm that makes at most $O(256^d)$ mistakes. Our proof is significantly simpler and uses only very basic properties of the Littlestone dimension. We also show that there exists no algorithm in this model that makes less than $3^d$ mistakes.
△ Less
Submitted 6 February, 2024; v1 submitted 15 August, 2023;
originally announced August 2023.
-
Inapproximability of sufficient reasons for decision trees
Authors:
Alexander Kozachinskiy
Abstract:
In this note, we establish the hardness of approximation of the problem of computing the minimal size of a $δ$-sufficient reason for decision trees.
In this note, we establish the hardness of approximation of the problem of computing the minimal size of a $δ$-sufficient reason for decision trees.
△ Less
Submitted 5 April, 2023;
originally announced April 2023.
-
Three iterations of $(d-1)$-WL test distinguish non isometric clouds of $d$-dimensional points
Authors:
Valentino Delle Rose,
Alexander Kozachinskiy,
Cristóbal Rojas,
Mircea Petrache,
Pablo Barceló
Abstract:
The Weisfeiler--Lehman (WL) test is a fundamental iterative algorithm for checking isomorphism of graphs. It has also been observed that it underlies the design of several graph neural network architectures, whose capabilities and performance can be understood in terms of the expressive power of this test. Motivated by recent developments in machine learning applications to datasets involving thre…
▽ More
The Weisfeiler--Lehman (WL) test is a fundamental iterative algorithm for checking isomorphism of graphs. It has also been observed that it underlies the design of several graph neural network architectures, whose capabilities and performance can be understood in terms of the expressive power of this test. Motivated by recent developments in machine learning applications to datasets involving three-dimensional objects, we study when the WL test is {\em complete} for clouds of euclidean points represented by complete distance graphs, i.e., when it can distinguish, up to isometry, any arbitrary such cloud.
Our main result states that the $(d-1)$-dimensional WL test is complete for point clouds in $d$-dimensional Euclidean space, for any $d\ge 2$, and that only three iterations of the test suffice. Our result is tight for $d = 2, 3$. We also observe that the $d$-dimensional WL test only requires one iteration to achieve completeness.
△ Less
Submitted 28 March, 2023; v1 submitted 22 March, 2023;
originally announced March 2023.
-
Find a witness or shatter: the landscape of computable PAC learning
Authors:
Valentino Delle Rose,
Alexander Kozachinskiy,
Cristobal Rojas,
Tomasz Steifer
Abstract:
This paper contributes to the study of CPAC learnability -- a computable version of PAC learning -- by solving three open questions from recent papers. Firstly, we prove that every improperly CPAC learnable class is contained in a class which is properly CPAC learnable with polynomial sample complexity. This confirms a conjecture by Agarwal et al (COLT 2021). Secondly, we show that there exists a…
▽ More
This paper contributes to the study of CPAC learnability -- a computable version of PAC learning -- by solving three open questions from recent papers. Firstly, we prove that every improperly CPAC learnable class is contained in a class which is properly CPAC learnable with polynomial sample complexity. This confirms a conjecture by Agarwal et al (COLT 2021). Secondly, we show that there exists a decidable class of hypothesis which is properly CPAC learnable, but only with uncomputably fast growing sample complexity. This solves a question from Sterkenburg (COLT 2022). Finally, we construct a decidable class of finite Littlestone dimension which is not improperly CPAC learnable, strengthening a recent result of Sterkenburg (2022) and answering a question posed by Hasrati and Ben-David (ALT 2023). Together with previous work, our results provide a complete landscape for the learnability problem in the CPAC setting.
△ Less
Submitted 23 February, 2023; v1 submitted 5 February, 2023;
originally announced February 2023.
-
Constant-Depth Sorting Networks
Authors:
Natalia Dobrokhotova-Maikova,
Alexander Kozachinskiy,
Vladimir Podolskii
Abstract:
In this paper, we address sorting networks that are constructed from comparators of arity $k > 2$. That is, in our setting the arity of the comparators -- or, in other words, the number of inputs that can be sorted at the unit cost -- is a parameter. We study its relationship with two other parameters -- $n$, the number of inputs, and $d$, the depth.
This model received considerable attention. P…
▽ More
In this paper, we address sorting networks that are constructed from comparators of arity $k > 2$. That is, in our setting the arity of the comparators -- or, in other words, the number of inputs that can be sorted at the unit cost -- is a parameter. We study its relationship with two other parameters -- $n$, the number of inputs, and $d$, the depth.
This model received considerable attention. Partly, its motivation is to better understand the structure of sorting networks. In particular, sorting networks with large arity are related to recursive constructions of ordinary sorting networks. Additionally, studies of this model have natural correspondence with a recent line of work on constructing circuits for majority functions from majority gates of lower fan-in.
Motivated by these questions, we obtain the first lower bounds on the arity of constant-depth sorting networks. More precisely, we consider sorting networks of depth $d$ up to 4, and determine the minimal $k$ for which there is such a network with comparators of arity $k$. For depths $d=1,2$ we observe that $k=n$. For $d=3$ we show that $k = \lceil \frac n2 \rceil$. For $d=4$ the minimal arity becomes sublinear: $k = Θ(n^{2/3})$. This contrasts with the case of majority circuits, in which $k = O(n^{2/3})$ is achievable already for depth $d=3$.
△ Less
Submitted 17 August, 2022;
originally announced August 2022.
-
Infinite Separation between General and Chromatic Memory
Authors:
Alexander Kozachinskiy
Abstract:
In this paper, we construct a winning condition $W$ over a finite set of colors such that, first, every finite arena has a strategy with 2 states of general memory which is optimal w.r.t.~$W$, and second, there exists no $k$ such that every finite arena has a strategy with $k$ states of chromatic memory which is optimal w.r.t.~$W$.
In this paper, we construct a winning condition $W$ over a finite set of colors such that, first, every finite arena has a strategy with 2 states of general memory which is optimal w.r.t.~$W$, and second, there exists no $k$ such that every finite arena has a strategy with $k$ states of chromatic memory which is optimal w.r.t.~$W$.
△ Less
Submitted 22 December, 2023; v1 submitted 4 August, 2022;
originally announced August 2022.
-
Energy Games over Totally Ordered Groups
Authors:
Alexander Kozachinskiy
Abstract:
Kopczyński (ICALP 2006) conjectured that prefix-independent half-positional winning conditions are closed under finite unions. We refute this conjecture over finite arenas. For that, we introduce a new class of prefix-independent bi-positional winning conditions called energy conditions over totally ordered groups. We give an example of two such conditions whose union is not half-positional. We al…
▽ More
Kopczyński (ICALP 2006) conjectured that prefix-independent half-positional winning conditions are closed under finite unions. We refute this conjecture over finite arenas. For that, we introduce a new class of prefix-independent bi-positional winning conditions called energy conditions over totally ordered groups. We give an example of two such conditions whose union is not half-positional. We also conjecture that every prefix-independent bi-positional winning condition coincides with some energy condition over a totally ordered group on periodic sequences.
△ Less
Submitted 16 August, 2022; v1 submitted 10 May, 2022;
originally announced May 2022.
-
State Complexity of Chromatic Memory in Infinite-Duration Games
Authors:
Alexander Kozachinskiy
Abstract:
A major open problem in the area of infinite-duration games is to characterize winning conditions that are determined in finite-memory strategies. Infinite-duration games are usually studied over edge-colored graphs, with winning conditions that are defined in terms of sequences of colors. In this paper, we investigate a restricted class of finite-memory strategies called chromatic finite-memory s…
▽ More
A major open problem in the area of infinite-duration games is to characterize winning conditions that are determined in finite-memory strategies. Infinite-duration games are usually studied over edge-colored graphs, with winning conditions that are defined in terms of sequences of colors. In this paper, we investigate a restricted class of finite-memory strategies called chromatic finite-memory strategies. While general finite-memory strategies operate with sequences of edges of a game graph, chromatic finite-memory strategies observe only colors of these edges.
Recent results in this area show that studying finite-memory determinacy is more tractable when we restrict ourselves to chromatic strategies. On the other hand, as was shown by Le Roux (CiE 2020), determinacy in general finite-memory strategies implies determinacy in chromatic finite-memory strategies. Unfortunately, this result is quite inefficient in terms of the state complexity: to replace a winning strategy with few states of general memory, we might need much more states of chromatic memory. The goal of the present paper is to find out the exact state complexity of this transformation.
For every winning condition and for every game graph with $n$ nodes we show the following: if this game graph has a winning strategy with $q$ states of general memory, then it also has a winning strategy with $(q + 1)^n$ states of chromatic memory. We also show that this bound is almost tight. For every $q$ and $n$, we construct a winning condition and a game graph with $n + O(1)$ nodes, where one can win with $q$ states of general memory, but not with $q^n - 1$ states of chromatic memory.
△ Less
Submitted 23 January, 2022;
originally announced January 2022.
-
New Bounds for the Flock-of-Birds Problem
Authors:
Alexander Kozachinskiy
Abstract:
In this paper, we continue a line of work on obtaining succinct population protocols for Presburger-definable predicates. More specifically, we focus on threshold predicates. These are predicates of the form $n\ge d$, where $n$ is a free variable and $d$ is a constant.
For every $d$, we establish a 1-aware population protocol for this predicate with $\log_2 d + \min\{e, z\} + O(1)$ states, where…
▽ More
In this paper, we continue a line of work on obtaining succinct population protocols for Presburger-definable predicates. More specifically, we focus on threshold predicates. These are predicates of the form $n\ge d$, where $n$ is a free variable and $d$ is a constant.
For every $d$, we establish a 1-aware population protocol for this predicate with $\log_2 d + \min\{e, z\} + O(1)$ states, where $e$ (resp., $z$) is the number of $1$'s (resp., $0$'s) in the binary representation of $d$ (resp., $d - 1$). This improves upon an upper bound $4\log_2 d + O(1)$ due to Blondin et al. We also show that any 1-aware protocol for our problem must have at least $\log_2(d)$ states. This improves upon a lower bound $\log_3 d$ due to Blondin et al.
△ Less
Submitted 9 May, 2022; v1 submitted 24 October, 2021;
originally announced October 2021.
-
One-to-Two-Player Lifting for Mildly Growing Memory
Authors:
Alexander Kozachinskiy
Abstract:
We investigate a phenomenon of "one-to-two-player lifting" in infinite-duration two-player games on graphs with zero-sum objectives. More specifically, let $C$ be a class of strategies. It turns out that in many cases, to show that all two-player games on graphs with a given payoff function are determined in $C$, it is sufficient to do so for one-player games. That is, in many cases the determinac…
▽ More
We investigate a phenomenon of "one-to-two-player lifting" in infinite-duration two-player games on graphs with zero-sum objectives. More specifically, let $C$ be a class of strategies. It turns out that in many cases, to show that all two-player games on graphs with a given payoff function are determined in $C$, it is sufficient to do so for one-player games. That is, in many cases the determinacy in $C$ can be "lifted" from one-player games to two-player games. Namely, Gimbert and Zielonka~(CONCUR 2005) have shown this for the class of positional strategies. Recently, Bouyer et al. (CONCUR 2020) have extended this to the classes of arena-independent finite-memory strategies. Informally, these are finite-memory strategies that use the same way of storing memory in all game graphs.
In this paper, we put the lifting technique into the context of memory complexity. The memory complexity of a payoff function measures, how many states of memory we need to play optimally in game graphs with up to $n$ nodes, depending on $n$. Now, assume that we know the memory complexity of our payoff function in one-player games. Then what can be said about its memory complexity in two-player games? In particular, when is it finite?
Previous one-to-two-player lifting theorems only cover the case when the memory complexity is $O(1)$. In turn, we obtain the following results. Assume that the memory complexity in one-player games is sublinear in $n$ on some infinite subsequence. Then the memory complexity in two-player games is finite.
We provide an example in which (a) the memory complexity in one-player games is linear in $n$; (b) the memory complexity in two-player games is infinite. Thus, we obtain the exact barrier for the one-to-two-player lifting theorems.
△ Less
Submitted 12 October, 2022; v1 submitted 28 April, 2021;
originally announced April 2021.
-
Continuous Positional Payoffs
Authors:
Alexander Kozachinskiy
Abstract:
What payoffs are positionally determined for deterministic two-player antagonistic games on finite directed graphs? In this paper we study this question for payoffs that are continuous. The main reason why continuous positionally determined payoffs are interesting is that they include the multi-discounted payoffs.
We show that for continuous payoffs, positional determinacy is equivalent to a simp…
▽ More
What payoffs are positionally determined for deterministic two-player antagonistic games on finite directed graphs? In this paper we study this question for payoffs that are continuous. The main reason why continuous positionally determined payoffs are interesting is that they include the multi-discounted payoffs.
We show that for continuous payoffs, positional determinacy is equivalent to a simple property called prefix-monotonicity. We provide three proofs of it, using three major techniques of establishing positional determinacy -- inductive technique, fixed point technique and strategy improvement technique. A combination of these approaches provides us with better understanding of the structure of continuous positionally determined payoffs as well as with some algorithmic results.
△ Less
Submitted 2 August, 2023; v1 submitted 16 December, 2020;
originally announced December 2020.
-
Polyhedral value iteration for discounted games and energy games
Authors:
Alexander Kozachinskiy
Abstract:
We present a deterministic algorithm, solving discounted games with $n$ nodes in $n^{O(1)}\cdot (2 + \sqrt{2})^n$-time. For bipartite discounted games our algorithm runs in $n^{O(1)}\cdot 2^n$-time. Prior to our work no deterministic algorithm running in time $2^{o(n\log n)}$ regardless of the discount factor was known.
We call our approach polyhedral value iteration. We rely on a well-known fac…
▽ More
We present a deterministic algorithm, solving discounted games with $n$ nodes in $n^{O(1)}\cdot (2 + \sqrt{2})^n$-time. For bipartite discounted games our algorithm runs in $n^{O(1)}\cdot 2^n$-time. Prior to our work no deterministic algorithm running in time $2^{o(n\log n)}$ regardless of the discount factor was known.
We call our approach polyhedral value iteration. We rely on a well-known fact that the values of a discounted game can be found from the so-called optimality equations. In the algorithm we consider a polyhedron obtained by relaxing optimality equations. We iterate points on the border of this polyhedron by moving each time along a carefully chosen shift as far as possible. This continues until the current point satisfies optimality equations.
Our approach is heavily inspired by a recent algorithm of Dorfman et al. (ICALP 2019) for energy games. For completeness, we present their algorithm in terms of polyhedral value iteration. Our exposition, unlike the original algorithm, does not require edge weights to be integers and works for arbitrary real weights.
△ Less
Submitted 25 October, 2020; v1 submitted 16 July, 2020;
originally announced July 2020.
-
Multiparty Karchmer-Wigderson Games and Threshold Circuits
Authors:
Alexander Kozachinskiy,
Vladimir Podolskii
Abstract:
We suggest a generalization of Karchmer-Wigderson communication games to the multiparty setting. Our generalization turns out to be tightly connected to circuits consisting of threshold gates. This allows us to obtain new explicit constructions of such circuits for several functions. In particular, we provide an explicit (polynomial-time computable) log-depth monotone formula for Majority function…
▽ More
We suggest a generalization of Karchmer-Wigderson communication games to the multiparty setting. Our generalization turns out to be tightly connected to circuits consisting of threshold gates. This allows us to obtain new explicit constructions of such circuits for several functions. In particular, we provide an explicit (polynomial-time computable) log-depth monotone formula for Majority function, consisting only of 3-bit majority gates and variables. This resolves a conjecture of Cohen et al. (CRYPTO 2013).
△ Less
Submitted 18 February, 2020;
originally announced February 2020.
-
An application of communication complexity, Kolmogorov complexity and extremal combinatorics to parity games
Authors:
Alexander Kozachinskiy,
Mikhail Vyalyi
Abstract:
So-called separation automata are in the core of several recently invented quasi-polynomial time algorithms for parity games. An explicit $q$-state separation automaton implies an algorithm for parity games with running time polynomial in $q$. It is open whether a polynomial-state separation automaton exists. A positive answer will lead to a polynomial-time algorithm for parity games, while a nega…
▽ More
So-called separation automata are in the core of several recently invented quasi-polynomial time algorithms for parity games. An explicit $q$-state separation automaton implies an algorithm for parity games with running time polynomial in $q$. It is open whether a polynomial-state separation automaton exists. A positive answer will lead to a polynomial-time algorithm for parity games, while a negative answer will at least demonstrate impossibility to construct such an algorithm using separation approach.
In this work we prove exponential lower bound for a restricted class of separation automata. Our technique combines communication complexity and Kolmogorov complexity. One of our technical contributions belongs to extremal combinatorics. Namely, we prove a new upper bound on the product of sizes of two families of sets with small pairwise intersection.
△ Less
Submitted 26 September, 2019; v1 submitted 19 February, 2019;
originally announced February 2019.
-
From expanders to hitting distributions and simulation theorems
Authors:
Alexander Kozachinskiy
Abstract:
Recently, Chattopadhyay et al. (\cite{chattopadhyay2017simulation}) proved that any gadget having so called \emph{hitting distributions} admits deterministic "query-to-communication" simulation theorem. They applied this result to Inner Product, Gap Hamming Distance and Indexing Function. They also demonstrated that previous works used hitting distributions implicitly (\cite{goos2015deterministic}…
▽ More
Recently, Chattopadhyay et al. (\cite{chattopadhyay2017simulation}) proved that any gadget having so called \emph{hitting distributions} admits deterministic "query-to-communication" simulation theorem. They applied this result to Inner Product, Gap Hamming Distance and Indexing Function. They also demonstrated that previous works used hitting distributions implicitly (\cite{goos2015deterministic} for Indexing Function and \cite{wu2017raz} for Inner Product).
In this paper we show that any expander in which any two distinct vertices have at most one common neighbor can be transformed into a gadget possessing good hitting distributions. We demonstrate that this result is applicable to affine plane expanders and to Lubotzky-Phillips-Sarnak construction of Ramanujan graphs . In particular, from affine plane expanders we extract a gadget achieving the best known trade-off between the arity of outer function and the size of gadget. More specifically, when this gadget has $k$ bits on input, it admits a simulation theorem for all outer function of arity roughly $2^{k/2}$ or less (the same was also known for $k$-bit Inner Product, (\cite{chattopadhyay2017simulation})). In addition we show that, unlike Inner Product, underlying hitting distributions in our new gadget are "polynomial-time listable" in the sense that their supports can be written down in time $2^{O(k)}$, i.e, in time polynomial in size of gadget's matrix.
△ Less
Submitted 6 July, 2018; v1 submitted 12 February, 2018;
originally announced February 2018.
-
Recognizing Read-Once Functions from Depth-Three Formulas
Authors:
Alexander Kozachinskiy
Abstract:
Consider the following decision problem: for a given monotone Boolean function $f$ decide, whether $f$ is read-once. For this problem, it is essential how the input function $f$ is represented. Our contribution consists of the following two results. We show that we can test in polynomial-time whether a given expression $C\lor D$ computes a read-once function, provided that $C$ is a read-once monot…
▽ More
Consider the following decision problem: for a given monotone Boolean function $f$ decide, whether $f$ is read-once. For this problem, it is essential how the input function $f$ is represented. Our contribution consists of the following two results. We show that we can test in polynomial-time whether a given expression $C\lor D$ computes a read-once function, provided that $C$ is a read-once monotone CNF and $D$ is a read-once monotone DNF and all the variables of $C$ occur also in $D$ (recall that due to Gurvich, the problem is coNP-complete when $C$ is read-2). The second result states that this is a coNP-complete problem to decide whether the expression $Ψ\land D_n$ computes a read-once function, where $D_n$ is as above and $Ψ$ is the $\bigwedge-\bigvee-\bigwedge$ depth-3 read-once monotone Boolean formula (so that the entire expression $Ψ\land D_n$ is depth-3 read-2). This result improves the result of \cite{elbassioni2011readability} in the depth and the result of \cite{gurvich2010it} in the readability of the input formula.
△ Less
Submitted 9 July, 2018; v1 submitted 11 February, 2018;
originally announced February 2018.
-
Automatic Kolmogorov complexity, normality and finite state dimension revisited
Authors:
Alexander Kozachinskiy,
Alexander Shen
Abstract:
It is well known that normality can be described as incompressibility via finite automata. Still the statement and the proof of this result as given by Becher and Heiber (2013) in terms of "lossless finite-state compressors" do not follow the standard scheme of Kolmogorov complexity definition (an automaton is used for compression, not decompression). We modify this approach to make it more simila…
▽ More
It is well known that normality can be described as incompressibility via finite automata. Still the statement and the proof of this result as given by Becher and Heiber (2013) in terms of "lossless finite-state compressors" do not follow the standard scheme of Kolmogorov complexity definition (an automaton is used for compression, not decompression). We modify this approach to make it more similar to the traditional Kolmogorov complexity theory (and simpler) by explicitly defining the notion of automatic Kolmogorov complexity and using its simple properties.
Using this characterization and a sufficient condition for normality in terms of Kolmogorov complexity derived from it, we provide easy proofs for classical results about normal sequences (Champernown, Wall, Piatetski-Shapiro, Besicovitch, Copeland, Erdos et al.)
Then we extend this approach to finite state dimension. We show that the block entropy definition of the finite state dimension remains the same if non-aligned blocks are used. Then we provide equivalent definitions in terms of automatic complexity, superadditive bounds for Kolmogorov complexity, calibrated superadditive functions and finite state a priori probability and use them to give simple proofs for known results about finite state dimension, and for Agafonov's result saying that normality is preserved by automatic selection rules as well as the results of Schnorr and Stimm that relate normality to finite state martingales.
Some results of this paper were presented at the Fundamentals in Computing Theory conferences in 2017 and 2019. Preliminary version of this paper (that does not mention the finite state dimension) was published in arxiv in~2017 (see the previous version of this submission).
△ Less
Submitted 24 August, 2020; v1 submitted 31 January, 2017;
originally announced January 2017.
-
Some Bounds on Communication Complexity of Gap Hamming Distance
Authors:
Alexander Kozachinskiy
Abstract:
In this paper we obtain some bounds on communication complexity of Gap Hamming Distance problem ($\mathsf{GHD}^n_{L, U}$): Alice and Bob are given binary string of length $n$ and they are guaranteed that Hamming distance between their inputs is either $\le L$ or $\ge U$ for some $L < U$. They have to output 0, if the first inequality holds, and 1, if the second inequality holds.
In this paper we…
▽ More
In this paper we obtain some bounds on communication complexity of Gap Hamming Distance problem ($\mathsf{GHD}^n_{L, U}$): Alice and Bob are given binary string of length $n$ and they are guaranteed that Hamming distance between their inputs is either $\le L$ or $\ge U$ for some $L < U$. They have to output 0, if the first inequality holds, and 1, if the second inequality holds.
In this paper we study the communication complexity of $\mathsf{GHD}^n_{L, U}$ for probabilistic protocols with one-sided error and for deterministic protocols. Our first result is a protocol which communicates $O\left(\left(\frac{s}{U}\right)^\frac{1}{3} \cdot n\log n\right)$ bits and has one-sided error probability $e^{-s}$ provided $s \ge \frac{(L + \frac{10}{n})^3}{U^2}$.
Our second result is about deterministic communication complexity of $\mathsf{GHD}^n_{0,\, t}$. Surprisingly, it can be computed with logarithmic precision: $$\mathrm{D}(\mathsf{GHD}^n_{0,\, t}) = n - \log_2 V_2\left(n, \left\lfloor\frac{t}{2}\right\rfloor\right) + O(\log n),$$ where $V_2(n, r)$ denotes the size of Hamming ball of radius $r$.
As an application of this result for every $c < 2$ we prove a $Ω\left(\frac{n(2 - c)^2}{p}\right)$ lower bound on the space complexity of any $c$-approximate deterministic $p$-pass streaming algorithm for computing the number of distinct elements in a data stream of length $n$ with tokens drawn from the universe $U = \{1, 2, \ldots, n\}$. Previously that lower bound was known for $c < \frac{3}{2}$ and for $c < 2$ but with larger $|U|$.
△ Less
Submitted 1 December, 2015; v1 submitted 27 November, 2015;
originally announced November 2015.
-
On Slepian--Wolf Theorem with Interaction
Authors:
Alexander Kozachinskiy
Abstract:
In this paper we study interactive "one-shot" analogues of the classical Slepian-Wolf theorem. Alice receives a value of a random variable $X$, Bob receives a value of another random variable $Y$ that is jointly distributed with $X$. Alice's goal is to transmit $X$ to Bob (with some error probability $\varepsilon$). Instead of one-way transmission, which is studied in the classical coding theory,…
▽ More
In this paper we study interactive "one-shot" analogues of the classical Slepian-Wolf theorem. Alice receives a value of a random variable $X$, Bob receives a value of another random variable $Y$ that is jointly distributed with $X$. Alice's goal is to transmit $X$ to Bob (with some error probability $\varepsilon$). Instead of one-way transmission, which is studied in the classical coding theory, we allow them to interact. They may also use shared randomness.
We show, that Alice can transmit $X$ to Bob in expected $H(X|Y) + 2\sqrt{H(X|Y)} + O(\log_2\left(\frac{1}{\varepsilon}\right))$ number of bits. Moreover, we show that every one-round protocol $π$ with information complexity $I$ can be compressed to the (many-round) protocol with expected communication about $I + 2\sqrt{I}$ bits. This improves a result by Braverman and Rao \cite{braverman2011information}, where they had $5\sqrt{I}$. Further, we show how to solve this problem (transmitting $X$) using $3H(X|Y) + O(\log_2\left(\frac{1}{\varepsilon}\right))$ bits and $4$ rounds on average. This improves a result of~\cite{brody2013towards}, where they had $4H(X|Y) + O(\log1/\varepsilon)$ bits and 10 rounds on average.
In the end of the paper we discuss how many bits Alice and Bob may need to communicate on average besides $H(X|Y)$. The main question is whether the upper bounds mentioned above are tight. We provide an example of $(X, Y)$, such that transmission of $X$ from Alice to Bob with error probability $\varepsilon$ requires $H(X|Y) + Ω\left(\log_2\left(\frac{1}{\varepsilon}\right)\right)$ bits on average.
△ Less
Submitted 8 September, 2015; v1 submitted 1 June, 2015;
originally announced June 2015.
