Scribd
Upload
35 views33 pages

Alt 09 Slides

This document summarizes key points about learning boolean functions from an algebraic perspective using monoids: 1) Certain classes of boolean functions computed by programs over monoids like Abelian groups and Gp ? Abelian groups can be learned from membership queries alone in polynomial time. 2) Decision lists having constant-degree polynomials over finite fields at the nodes, which corresponds to programs over aperiodic monoids and finite p-groups, can be learned from equivalence queries alone in polynomial time. 3) The class of boolean functions computed by multiplicity automata over finite fields, which includes polynomials and unambiguous DNF, can be learned from both membership and equivalence queries in polynomial time. This corresponds to programs over languages

Uploaded by

Kevin Mondragon
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
35 views33 pages

Alt 09 Slides

This document summarizes key points about learning boolean functions from an algebraic perspective using monoids: 1) Certain classes of boolean functions computed by programs over monoids like Abelian groups and Gp ? Abelian groups can be learned from membership queries alone in polynomial time. 2) Decision lists having constant-degree polynomials over finite fields at the nodes, which corresponds to programs over aperiodic monoids and finite p-groups, can be learned from equivalence queries alone in polynomial time. 3) The class of boolean functions computed by multiplicity automata over finite fields, which includes polynomials and unambiguous DNF, can be learned from both membership and equivalence queries in polynomial time. This corresponds to programs over languages

Uploaded by

Kevin Mondragon
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 33

An Algebraic Perspective on Boolean Function Learning

Ricard Gavaldà1 Denis Thérien2


1 LARCA research group
Univ. Politècnica de Catalunya
Barcelona, Spain
2 Schoolof Computer Science
McGill University
Montréal, Canada

ALT’09, Porto, october 5th 2009


Introduction

We can learn boolean functions represented in many ways:

Conjunctions, k -CNF, k -DNF, monotone DNF, Deterministic


Finite Automata, k -term DNF, k -decision lists, read-once formulas,
bounded rank decision trees, constant-degree polynomials, sparse
polynomials, threshold gates, decision trees, CDNF formulas,
multisymmetric concepts, conjunctions of Horn clauses, O(log n)-term DNF,
nested subspaces, counter languages, OBDD, Multiplicity (Weighted) Automata, . . .
Introduction

Programs over monoids


. . . yet another representation of boolean functions!!

yes, but

gives context: detailed, deep taxonomies of monoids


highlights a few unnoticed learnable classes
suggests limits of current techniques
Summary

Membership queries: algorithm for MOD p ◦ MOD m circuits


Equivalence queries: decision lists over constant-degree
polynomials over Fp
Membership + Equivalence:
Maximal class of functions learnable as Multiplicity
Automata

Unifies many known results


Does not capture: monotonicity, threshold circuits, read-k
conditions, sensitivity to variable ordering
Background: Algebra and circuits
Semigroups

A semigroup is set with a binary, associative operation


A monoid is a semigroup with an identity
A group is a monoid where each element has an inverse

A monoid A divides a monoid B if A is a homomorphic


image of a subsemigroup of B
An aperiodic (aka group-free) monoid is one that is divided
by no nontrivial group
Monoid products

The direct product of A and B, A × B is defined by

(a1 , b1 ) · (a2 , b2 ) = (a1 · a2 , b1 · b2 )

A semidirect product of A and B is defined by choosing a


function f : A × B → A and

(a1 , b1 ) · (a2 , b2 ) = (a1 · f (a2 , b1 ), b1 · b2 )

The wreath product of A and B, denoted A ? B, generalizes


semidirect product by accounting for all choices of f
Decomposition theorem

Theorem [Krohn-Rhodes 62]


1. Every finite semigroup M divides a wreath product of finite
simple groups and copies of the flip-flop monoid∗

2. Only finite simple groups are required if M is a group

3. Only flip-flop monoids are required if M is aperiodic

∗ A particular 3-element aperiodic monoid


Boolean functions

Functions f : {0, 1}n → {0, 1}


AND, OR, NOT, threshold gates
Generalized MOD m gates
n
A
MODm (x1 , . . . , xn ) = 1 iff ( ∑ xi ) ∈ A
i=1

Decision lists, decision trees


Deterministic Finite Automata
Weighted Automata or Multiplicity Automata over rings
M(x1 , . . . , xn ) = sum over all paths consistent with x1 . . . xn
of product of labels in path
Programs over monoids

An instruction over a monoid M is a triple (i, u, v )


Interpreted as “read xi and emit u if xi = 0, v if xi = 1”

A program over M is a sequence of instructions


L = (I1 , . . . , Is ) plus an accepting set A ⊆ M
 s
1 if
∏ Ii (x) ∈ A,

(L, A)(x) = i=1

0 otherwise.
Programs over monoids (2)

Each program P over M computes a boolean function B(P)


B(M) is the set of boolean functions computed by
programs over M
For a class of monoids M
[
B(M ) = B(M)
M∈M
From monoids to boolean functions

Division: If M1 divides M2 then B(M1 ) ⊆ B(M2 )

Direct product:

B(M1 × M2 ) ≡ boolean combinations of B(M1 ) and B(M2 )


≡ NC 0 ◦ (B(M1 ) ∪ B(M2 ))

Wreath product: For G a group,

B(M ? G) = B(M) ◦ B(G)


Examples

Classical examples [Barrington 87, Barrington-Thérien 89]:

Monoidland Circuitland
all monoids NC 1
any nonsolvable group NC 1
Abelian groups boolean combinations of MOD gates
solvable groups poly-size, constant-depth circuits
made of MOD gates
aperiodic monoids poly-size constant-depth circuits
made of AND, OR, NOT gates

For learning we should remain well below NC 1


Dramatis personae, groups

Group Description

Abelian groups direct products cyclic groups

Gp or p-groups groups of cardinality pk

Nilpotent groups direct products of p-groups

Solvable groups wreath product of cyclic groups


Dramatis personae, groups

Groupland Circuitland
Abelian groups MOD m ,
degree 1 polynomials over Zm
Gp or p-groups MOD pk ◦ NC 0 , MOD p ◦ NC 0 ,
constant degree polynomials over Fp
Nilpotent groups MOD m ◦ NC 0 ,
constant degree polynomials over Zm
Solvable groups constant-depth, poly-size modular circuits
Dramatis personae, aperiodic

DA monoids: (stu)k t (stu)k = (stu)k for some k

In circuitland [GT03]:

[
B(DA) = rank-k decision trees
k
B(DA) ◦ NC 0 =
[
k -decision lists
k

Borderline of expressivity in several contexts (descriptive


complexity, communication complexity)
(Almost) nothing between B(DA) and DNF, in monoidland
Membership queries
Negative results

Fact [GT06]
Learning programs over M requires 2n Membership queries if
M is not a group
or M is a nonsolvable group

Reason: Can compute singletons in polynomial size


What about solvable groups?

Two subclasses of solvable groups provably weaker than NC 1 :

Nilpotent groups
Equivalent to polynomials of constant degree over some Zm
Includes Abelian groups and Gp

Gp ? Abelian
Equivalent to depth-2, MOD p -of-MOD m circuits
Group lower bounds

If G nilpotent, any two programs of length s over G differ on


some assignment of weight cG [PT88]

If G ∈ Gp ? Abelian, any two programs of length s over G differ


on some assignment of weight cG log s [BST89]

Learning strategy:
1 Ask Membership queries with all assignments of weight cG
(or weight cG log s)
2 Build unique program consistent with the answers
Part 2 is a purely computational problem
Abelian groups

Theorem
If G is Abelian, then B(G) is learnable from Membership
queries in nO(1) time

Equivalent to MODm gates and degree-1 polynomials over Zm

Open: extend to degree-O(1) polynomials (= nilpotent groups)


Gp ? Abelian

Theorem
If G ∈ Gp ? Abelian, then B(G) is learnable from Membership
queries in nO(log s) time

Equivalent to MODp -of-MODm circuits

Known to be learnable in time (n + s)O(1) from Membership and


Equivalence queries [BBTV97]
Equivalence queries
DL ◦ MOD p ◦ NC 0

Theorem [from known results]


Decision lists having constant-degree polynomials over Fp at
the nodes are learnable from nO(1) Equivalence queries

Combine:
Tricks to make MOD p gates 0/1-valued [Fermat,BT94]
Subspace learning algorithm [HSW87]
Decision list / nested difference algorithm [R87,HSW87]
Composition theorem
DL ◦ MOD p ◦ NC 0

DL ◦ MOD p ◦ NC 0 subsumes:
DL ◦ MOD p : nested differences of linear subspaces of Fp
DL ◦ NC 0 : k -DL, so rank-k DT’s, k -CNF and k -DNF
MOD p ◦ NC 0 : constant-degree polynomials over Fp
strict width-2 branching programs [BBTV97]

Note: All these classes are nonuniversal


Algebraic equivalent

Theorem
1. DL ◦ MOD p ◦ NC 0 = B(DA ? Gp )
0
m DL ◦ MOD m ◦ NC
S
2. = B(DA ? Nilpotent)

Hence B(DA ? Gp ) learnable from nO(1) Equivalence queries


With Equivalence queries, B(DA ? Abelian) learnable iff
B(DA ? Nilpotent) learnable
What’s the ceiling?

If M ∈ DA ? Gp then M is not universal

If M 6∈ DA ? Nilpotent then M is universal 1

For M in between, we don’t know; basic first question

1 subtle lie here; see proceedings


Membership and Equivalence queries
Multiplicity Automata

Theorem [BBBKV00]
Functions Σ∗ → Fp computed by Multiplicity Automata over Fp
are polynomial-time learnable from Membership and
Equivalence queries.

Subsumes, besides DFA:


polynomials over Fp
unambiguous DNF (hence decision trees and k -term DNF)
MODp -of-MODm circuits
Algebraic characterization

LGp
m Com [Weil 87]
The value of m1 . . . ms can be determined by counting mod p
the number of factorizations of the form a0 La1 La2 . . . ak −1 Lak ,
for L a commutative language (bool comb of)

Theorem
1. B(LGp
m Com) is polynomially simulated by MA over Fp

2. unambiguous DNF, polynomials, and MODp -of-MODq


circuits are polynomially simulated in B(LGp
m Com)
Order sensitivity

Conjecture
LGp
m Com is the largest class of monoids that is polynomially
simulated by MA

Intuition: If M 6∈ LGp m Com there is a function f ∈ B(M) such


that f has MA of size poly (n) but the smallest MA for some
f (xπ(1) , . . . , xπ(n) ) has size 2Ω(n)
There is an explicit characterization [TT07] of monoids not in
LGp
m Com
In summary
Conclusions

Many learning results can be unified into 3 algorithms for


learning large classes of monoids

Extending to larger classes seems to require either proving


new lower bounds or learning DNF

Open problem: Efficiently learn one MODm ◦ NC 0 gate

You might also like