Learning Syntax Without Planting Trees: Understanding When and Why Transformers Generalize Hierarchically

Given a sequence of tokens, the language modeling objective trains the model to predict each token in a sequence given the preceding tokens. The model is optimized to minimize the negative log-likelihood of the sequences in the training data. For transformers, the language modeling objective is typically associated with decoder-only models like GPT (Brown et al., 2020), and the loss is computed over all tokens in the sequence. For the question formation task and the declarative-question pair from the introduction, if $\boldsymbol{s}=\langle s_{1},s_{2},\ldots,s_{21}\rangle=\langle$my, walrus, does, move, the, dogs, that, do, wait, ., quest, does, my, walrus, move, the, dogs, that, do, wait, ?$\rangle$, the cross-entropy loss is computed over $s_{1}$ through $s_{21}$, each given the preceding tokens:

The sequence-to-sequence (seq2seq) modeling objective (Sutskever et al., 2014), is used to train the model to generate a target sequence (e.g., from the example above, $\langle s_{12},\ldots,s_{21}\rangle$) given an input sequence ($\langle s_{1},\ldots,s_{11}\rangle$). This objective, which includes only the terms from $i=12$ to $21$ in equation 1, is typically associated with an encoder-decoder model as used in the original transformer architecture (Vaswani et al., 2017). Note that the seq2seq objective is more suited for tasks with an explicit input and output (like question formation and tense inflection), but is not suitable for the simple agreement task. Hence, we do not evaluate the seq2seq objective for simple agreement.

In the prefix language modeling objective (Dong et al., 2019), we again generate the output text given the input (or “prefix”), but we use a single transformer decoder (similar to language modeling) instead of an encoder-decoder model. Differing from the original language modeling objective, here the loss is only computed over the output text and does not include the prefix. One modification that we make to how the prefix-LM objective is typically used, is that we use a causal mask for the prefix tokens as well instead of having bi-directional attention over the prefix tokens, since we found the latter to perform subpar in our initial experiments (unstable in-distribution performance).

In the sequence classification objective, the model is trained to map the entire sequence to a discrete label. E.g., for question formation the model is given the input declarative sentence and trained to predict the correct auxiliary from the set of auxiliary verbs (do, does, don’t, doesn’t) that should occur at the start of the question, i.e., a four-way classification task.

In the cloze completion setting, the model is given a sequence of tokens with some tokens masked and trained to predict the masked tokens. E.g., for the question formation task, we consider the declarative-interrogative pair and mask out tokens in the interrogative sentence at all positions where the auxiliaries could be present. Specifically, we have mask tokens where (i) the auxiliary is present in the interrogative sentence or (ii) the auxiliary was present in the original declarative sentence. The model is trained to predict the correct auxiliary at these positions and $<$EMPTY$>$ if an auxiliary is not present at a particular position. Note that this objective is similar to masked language modeling as in Devlin et al. (2019); however, instead of masking tokens randomly, we mask the specific tokens as described above.³³3Our initial experiments with random-masking resulted in subpar performance, even on in-distribution test sets. For the passivization task, we do not evaluate the cloze completion objective, because (unlike other tasks) the output sequence is significantly different from the input and not just in terms of one or two tokens, which makes defining the masking strategy in this case non-trivial.

Please refer to §A.1.1 for full details of each objective for all of the five tasks.

By design of the test datasets, a model following the linear rule will obtain 100% in-distribution accuracy and 0% generalization accuracy. Only a model consistent with the hierarchical rule will obtain 100% accuracy on both test sets for all the tasks.

Our results above suggest that the language modeling objective imposes bias towards hierarchical generalization in transformers. Based on this, we revisit hierarchical generalization in RNNs, as the experiments of McCoy et al. (2020) were conducted using the seq2seq objective (and found to not generalize when not using attention mechanism). We train 2-layer GRU (Cho et al., 2014) models using a learning rate of 0.001 using both LM and seq2seq objectives for the question formation task. As shown in Figure 2, like transformers, RNNs also generalize hierarchically on the question formation task when trained using the language modeling objective, but that’s not the case for the seq2seq objective. This further strengthens the evidence that language modeling acts as a source of inductive bias for hierarchical generalization.

Overall, our experiments implicate language modeling as a source of inductive bias for the neural models to generalise hierarchically. We hypothesize that the reason LMs (approximately) learn the hierarchical rule rather than the linear rule is that, when viewing the full training objective as modeling the distribution of all the tokens in the sentence pairs and not just moving of the auxiliary to the beginning, it is the combined simplicity of the hierarchical rule and the data is greater than the linear rule and the data. Perhaps modeling the hierarchical phrase structure is beneficial for modeling the distribution over full sequences. We will explore this hypothesis in more depth in §5.

Unless specified otherwise, for pruning, we use a learning rate of $0.05$, the $L_{0}$ regularization penalty coefficient as $0.015$, and train for 10k steps, which we found to work well across different pruning settings. Here we report the experiments for the question formation task and discuss the others in Appendix §A.2, for which we also obtain consistent results. Note that since we are interested in discovering subnetworks implementing hierarchical and linear rules, while pruning, we only use the negative $\log$-likelihood of the first auxiliary in the question for computing the loss. To make sure that the discovered subnetworks are not just a by-product of the pruning procedure, we also consider control groups, which are obtained by pruning randomly initialized networks.

We now consider how these different subnetworks evolve over the course of the model training. To this end, we save checkpoints after every 1000 steps of training and perform the three kinds of pruning on those checkpoints. As shown in Figure 4(b), the “linear-rule” subnetwork becomes discoverable through pruning at roughly 6000 training steps – indicated by the 0% generalization performance for $\texttt{Train}\backslash\texttt{Gen}$-prune curve and 100% in-distribution performance in Figure 4(a). Soon after, we see in Figure 4(b), the formation of a (hierarchical) generalization subnetwork, at around 9000 training steps, where the subnetwork obtained using Gen-prune obtains 100% generalization accuracy (the full network at this point only obtains 5% generalization performance). While there are occasional spikes in the generalization performance of subnetworks found using $\texttt{Train}\backslash\texttt{Gen}$-prune during the first-thirds of the training, during majority of the training (especially the latter two-thirds) both the linear and hierarchical generalization subnetworks continue to be discoverable over the course of training – as indicated by stable 100% generalization accuracy of subnetworks found by Gen-prune and 0% generalization accuracy for $\texttt{Train}\backslash\texttt{Gen}$-prune subnetworks in Figure 4(b) .

Our experiments reveal that, throughout training, there is competition between the two sub-networks, and while the behavior of the aggregate model becomes closer to the hierarchical rule with training, the competing linear-rule subnetwork does not really disappear. All three pruning methods are unsuccessful on the control group (randomly initialized networks), providing further evidence that these subnetworks are not introduced by the pruning methods, and behaviors akin to the hierarchical and linear rules are implemented within the language model.

We hypothesize that the ambiguous training data (with two plausible generalizations, linear and hierarchical) is the reason for the existence of the subnetworks with very different generalization behaviors. To evaluate this hypothesis, we consider the case where the model is trained with disambiguated data – we augment the ambiguous training dataset with examples that are only consistent with the hierarchical rule.⁷⁷7We add 10k such examples to the existing dataset containing 100k examples. We plot the training dynamics of the model trained with this disambiguated data in Figure 4(c). The full model without any pruning in this case, as expected, generalizes perfectly after a few thousand training steps (see 4(c)) without any noise, in contrast with generalization accuracy of the full-model trained on ambiguous data in Figure 4(b). More interestingly, Figure 4(c) shows that $\texttt{Train}\backslash\texttt{Gen}$-prune fails to yield subnetworks that obtain 0% generalization accuracy, in contrast to the ambiguous data case in figure 4(b). To make sure this is not due to the choice of our pruning hyperparameters, we conduct an extensive hyperparameter search, consisting of 128 combinations of the pruning learning rate, regularization penalty, and pruning steps (using Bayesian optimization) and still fail to find the setting where the $\texttt{Train}\backslash\texttt{Gen}$-prune succeeds for the disambiguated data case (see Figure 12 in Appendix). This strongly suggests that the “linear-rule” subnetwork is never formed in the language model when it doesn’t have a reason to be learned – when the alternate generalization behavior (linear rule) is no longer applicable to the entire training dataset.

We also experiment with the opposite disambiguation setup, where we augment the training data with examples only consistent with the linear rule, and in line with our findings, we find that the Gen-prune fails to find a subnetwork with 100% generalization accuracy – no subnetwork consistent with hierarchical rule is formed (Figure 13 in Appendix). Hence, ambiguity in the training data appears to drive the joint existence of these contrasting subnetworks.

We make use of Solomonoff’s theory of inductive inference (Solomonoff, 1964), which formalizes Occam’s razor – when two hypotheses explain the data equally well, the simpler one of the two is likely to be the correct one. This notion is mathematically formalised in Solomonoff’s theory using a Bayesian approach by computing the posterior probabilities of the competing hypotheses and selecting the one with higher posterior.

Here, $p(D\mid h)$ denotes the likelihood of the observed data $D$ based on the hypothesis $h$, i.e., how well $h$ explains the data $D$. $p(h)$ denotes the prior probability of $h$, which in Solomonoff’s theory assigned higher values for simpler hypotheses $h$. In other words, a more complex hypothesis will entail making more choices (“high program length”) and hence have a lower prior probability. Hence, by computing the posterior $p(h\mid D)$, Bayesian inference balances the tradeoff between the goodness of fit of a hypothesis (likelihood) and its simplicity (prior). This is closely related to “Bayesian Occam’s razor” and the Minimum Description Length principle (Rissanen, 1978).

We mentioned in the previous paragraph that computing the posterior over the competing hypotheses can help us choose the one which better balances the trade-off between goodness of fit and simplicity. But for our problem, what form should these hypotheses or “programs" take to represent the linear and hierarchical rules? Note that since our training objective is language modeling, we need to consider the hypotheses that generate the entire sequence of tokens as represented in the training data. Following Perfors et al. (2011), we use probabilistic generative grammars to model the data-generation process.

For the purposes of this work we consider probabilistic context-free grammars (PCFGs) that can be represented using a 5-tuple i.e., $G=\{V,\Sigma,R,S,P\}$. Here, $V$ denotes the set of nonterminal symbols that form phrases or constituents in a sentence, $\Sigma$ denotes the set of terminal symbols or words in the sentences, $R\in V\times\{V\cup\Sigma\}^{*}$ denotes the set of production rules mapping phrases to sub-phrases or words, $S\in V$ is the start symbol that represents the whole sentence, and $P$ denotes the probabilities on the production rules. Specifically, for a given non-terminal when there are multiple productions possible, $P$ assigns a probability to each possible production rule. To generate data from a PCFG, we start from the start symbol $S$ and for each terminal that arises we apply a production rule sampled according to $P$ and repeat the procedure till we are only left with terminals to generate sentences. PCFGs are typically used to model the hierarchical phrase structure of a language. We can also apply some constraints to the form of production rules in $R$ to obtain special cases (subsets) of CFGs. For example, regular grammars form a subset of CFGs, whose production rules can be put into a right-linear form: $A\to bC$, where $A$ and $C$ are nonterminal symbols and $b$ is a terminal.

We can view the data-generation process that generates dataset $D$ using the probabalistic grammar $G$ Given the dataset $D$, we can compute the posterior $p(G\mid D)\propto p(D\mid G)\cdot p(G)$, where $p(D\mid G)$ is the likelihood of the data given the probabilistic grammar, and $p(G)$ measures the simplicity of $G$. To get an intuitive understanding of the prior probability of a grammar and how it encodes simplicity, recall that grammars that we consider are 5-tuples $\{V,\Sigma,R,S,P\}$. Hence choosing a grammar $G$ involves making choices like the number of nonterminal and terminal symbols, number of production rules from each nonterminal, nature of the production rule etc. By assigning probability distributions to each of these choices, we can compute the prior probability of a given grammar. We can choose the prior distribution that favours simpler grammars, e.g., following Perfors et al. (2011), we use geometric distributions for the number of nonterminals and productions, hence a simple grammar with fewer nonterminals and productions will receive a higher probability compared to a more complex grammar. We can hence compute the posteriors for the grammars representing the competing generalization hypotheses (hierarchical and linear rule) to compare how each of these balances the tradeoff between goodness of fit and simplicity.

For the purposes of this study we consider the simple agreement task, as constructing hierarchical and linear grammars for its data is straightforward.⁹⁹9Question formation and tense reinflection involve pairs of sentences, where the second sentence is a transformed version of the first. Such sentence pairs would likely require more complex frameworks like synchronous grammars (Aho and Ullman, 1969), which we leave to future work.

Following Perfors et al. (2011), we hand-construct the CFG and regular grammars. The CFG is constructed so that each verb agrees with the hierarchically connected subject, while the regular grammar is constructed to follow the linear rule (each verb in the sentence agrees with the most recent noun). The constructed grammars are assigned uniform probabilities for the production rules i.e., given a nonterminal, all the productions are equally likely. For an example of productions from both the grammars see Figures 16 and 17 in the Appendix. For constructing CFG, we use Chomsky Normal Form for the productions: Each production rule is of the form $A\to BC$ or $A\to a$, where $A,B,C$ are nonterminals and $a$ is a terminal symbol. Similarly, for the regular grammar Reg, we use the right-linear form of productions: Every rule is of the form $A\to bC$ or $A\to a$.

Following Perfors et al. (2011), we adopt a type-based approach for constructing the grammars: terminal symbols $\Sigma$ instead of being the word tokens (e.g. walrus, sing) are syntactic categories (e.g., determiner, singular-noun, intransitive-verb, etc.), so that we can use these grammars to strictly model abstract syntactic structures and not vocabulary-type frequencies, and it also gives us a manageable number of possible generations by the grammars.

For both context-free and regular grammars we generate two variants, depending on the diversity of the sentence types generated by them:

Small grammars CFG-S and Reg-S: Here we construct CFG and regular grammars that only generate 18 sentence types. Recall that a sentence type is a sequence of syntactic categories, e.g., sentences like The walrus sings can be represented by sentence type determiner singular-noun intransitive-verb. Different sentence types in this case differ by the plurality of the nouns (singular or plural), type of verbs (transitive or intransitive), and presence or absence of prepositional phrases accompanying the nouns. The resulting hand-constructed CFG-S in this case has 15 nonterminals and 21 production rules and Reg-S has 14 nonterminals and 22 production rules. Both grammars have the same 8 terminals. Out of the 18 sentence types generated by both the grammars, 12 are common between the two (ambiguous) and 6 remaining in CFG-S that are only consistent with the hierarchical rule and 6 only consistent with linear rule in Reg-S.

Large grammars CFG-L and Reg-L: In this case we consider larger grammars, which can generate much more diverse sentence types – 180 sentence types. The major difference with the smaller grammars here is that they are allowed to generate relative clauses, which can be present at both the subject and object in the sentence. CFG-L has 25 nonterminals and 38 productions, while Reg-L has 41 nonterminals and 63 productions. Note that based on these numbers alone it is evident that we need much more complex regular grammars to generate diverse sentence types. Out of the 180 sentence types generated by each grammar, 120 are common between the two, and the remaining sentence types are only generated by the specific grammars (following either hierarchical or linear rule).

We generate the sentence types from each of the 4 grammars – $\mathcal{D}_{\texttt{CFG-S}}$, $\mathcal{D}_{\texttt{Reg-S}}$, $\mathcal{D}_{\texttt{CFG-L}}$, and $\mathcal{D}_{\texttt{Reg-L}}$. As mentioned before, the training dataset is constructed by considering the sentence types common between the CFG and corresponding regular grammar. We have $\mathcal{D}_{\mathrm{train-S}}{}=\mathcal{D}_{\texttt{CFG-S}}{}\cap\mathcal{D}_{\texttt{Reg-S}}{}$ for the small grammars, and $\mathcal{D}_{\mathrm{train-L}}{}=\mathcal{D}_{\texttt{CFG-L}}{}\cap\mathcal{D}_{\texttt{Reg-L}}{}$ for the larger ones. Note that these are the datasets of sentence-types, and transformers are trained on sentences. To generate sentences from these type corpora, we repeatedly sample sentence types, and replace the syntactic categories with the allowed tokens for that category (e.g., determiner can be replaced with the, our, my, etc.). Using this procedure we generate a corpus of 50k sentences from $\mathcal{D}_{\mathrm{train-S}}{}$ and 50k sentences from $\mathcal{D}_{\mathrm{train-L}}{}$. Note that the simple agreement experiments in §3.1, were performed using the latter dataset derived from $\mathcal{D}_{\mathrm{train-L}}{}$.

The generalization test sets are generated by considering the sentence types that are unique to a specific grammar. E.g., we can have the test set $\mathcal{D}_{\mathrm{test-S}}^{\texttt{Hier}}=\mathcal{D}_{\texttt{CFG-S}}\setminus\mathcal{D}_{\texttt{Reg-S}}$, which contains sentence types that are unique to $\mathcal{D}_{\texttt{CFG-S}}$ and hence only consistent with the hierarchical rule and not the linear rule. Similarly, $\mathcal{D}_{\mathrm{test-S}}^{\texttt{Lin}}=\mathcal{D}_{\texttt{Reg-S}}\setminus\mathcal{D}_{\texttt{CFG-S}}$, consists of sentence types consistent only with the linear rule. We can equivalently define $\mathcal{D}_{\mathrm{test-L}}^{\texttt{Hier}}$ and $\mathcal{D}_{\mathrm{test-L}}^{\texttt{Lin}}$. While talking about the two datasets in general and not specifically about the small ($S$) or large ($L$) variants, we just use the notation $\mathcal{D}_{\mathrm{test}}^{\texttt{Hier}}$ and $\mathcal{D}_{\mathrm{test}}^{\texttt{Lin}}$.

Now that we have the four grammars constructed, we can compute the posteriors for the grammars given the corresponding training datasets. Note that, since we are only interested in comparing the posteriors of CFG and regular grammars, we can estimate the posterior by computing the likelihood and prior and taking product of the two, i.e., $p(G|D)\propto p(D\mid G)\,p(G)$. Recall that the prior probability of a grammar can be computed by calculating the probability of each of the choices that goes into defining that grammar:

Here, $|V|$ is the number of nonterminals, $P_{k}$ is the number of productions from the $k^{\mathrm{th}}$ nonterminal with the probabilities of each production given by $\theta_{k}\in[0,1]^{P_{k}}$, and $R_{k,i}$ denotes the right hand side of the $i$th production rule from the $k$th nonterminal. Following, Perfors et al. (2011), we use a geometric prior on $p(|V|)$ and $p(P_{k})$. Recall that the geometric distribution is given by $p(n;p)=(1-p)^{n-1}p$, where $p$ is a parameter of the geometric distribution, often interpreted as the probability of success, and a geometric distribution models the probability of success after $n$ trials. Hence, choosing a geometric prior penalizes the grammars with a large number of nonterminals ($|V|$) and productions per nonterminal ($P_{k}$). In our experiments we use $p=0.5$, following Perfors et al. (2011), but we conduct a sensitivity analysis on the choice of this parameter . For $\theta_{k}$, we use a flat (i.e., $\alpha=1$) Dirichlet prior, a popular choice for modeling probabilities for categorical distributions ($K-1$ simplex). Note that since the Dirichlet is a continuous distribution, the probability of any specific $\theta_{k}$ is zero and we use the discrete relaxation from Perfors et al. (2011) to model $p(\theta_{k})$. The probability of the production rule $p(R_{k,i})$, depends on the type of grammar. For CFGs, since we consider them in CNF, the production rules are of the form $A\to BC$ or $A\to a$, hence the probability of the right hand side can be given by, $p(R_{k,i})=\frac{1}{2}\frac{1}{|V|^{2}}\mathds{1}(|R_{k,i}|=2)+\frac{1}{2}\frac{1}{|\Sigma|}\mathds{1}(|R_{k,i}|=1)$. Since the regular grammars are in the right linear form i.e. productions of the form $A\to bC$ or $A\to a$, we can compute $p(R_{k,i})=\frac{1}{2}\frac{1}{|\Sigma|}\frac{1}{|V|}\mathds{1}(|R_{k,i}|=2)+\frac{1}{2}\frac{1}{|\Sigma|}\mathds{1}(|R_{k,i}|=1)$. One might notice that we are missing the probability of number of terminal symbols $p(\Sigma)$ in the prior equation. We ignore this because both the CFG and regular grammars have the same number of terminals in our experiments, and since we are interested in just comparing the probabilities, the inclusion or exclusion of $p(\Sigma)$ doesn’t make a difference.¹⁰¹⁰10One might also notice that $p(G)$ allows some probability for generating the same rule more than once; it “leaks” probability mass. No prior literature, to our knowledge, suggests that this should pose a problem to our analysis.

The likelihood $p(D\mid G)$, measures the probability that the dataset $D$ is generated from the grammar $G$. For $m$ sentence types in the dataset $D$, the likelihood is given by

where $S_{i}$’s denote the sentence types in $D$. $p(S_{i}\mid G)$ is computed by taking product of the probabilities of production rules used to derive $S_{i}$ using $G$ (including adding the probabilities when multiple parses are possible for $S_{i}$). Note that computing $p(S_{i}\mid G)$ requires estimating the production probabilities $\theta_{k}$ from each nonterminal. We use the Inside-Outside algorithm (Baker, 1979), to obtain an approximate maximum likelihood estimate of the production probabilities on the dataset $D$. Hence, having computed both the prior $p(G)$ and $p(D\mid G)$, we can compute the posterior $p(G\mid D)$.

Given our generated training datasets ($\mathcal{D}_{\mathrm{train-S}},\mathcal{D}_{\mathrm{train-L}}$), there can be grammars other than the four we constructed that can generate these datasets. In their analysis, Perfors et al. (2011) also consider two subsets of the regular grammars: Flat and One-state. Flat grammars have production rules which are the list of memorized sentences, i.e., of the form $S\to a_{1}a_{2}\cdots a_{n}$. Here $a_{i}^{\prime}s$ are terminal symbols and there are no nonterminals other than $S$. Hence, flat grammars can be used to model memorization without generalization. One-state grammars are equivalent to finite state automata with a single state and hence permit any terminal symbol to follow any other. We also include these two grammars in our analysis.

Further, even among the class of context-free and regular grammars, there might exist grammars with better posteriors on the training datasets $\mathcal{D}_{\mathrm{train-S}}$ and $\mathcal{D}_{\mathrm{train-L}}$ than the ones that we hand-construct. To remedy this, we also experiment with applying local search on our constructed grammars, using Bayesian model merging (Stolcke and Omohundro, 1994) to minimize the grammars while improving the posterior on the respective training datasets. While in the main text we discuss the results for hand-constructed grammars, we provide the details on the minimization algorithm and the corresponding results for minimized grammars in §A.3.

Recall that our goal has been to quantify the notion of simplicity of the two competing hypotheses (hierarchical and linear rule), which are consistent with the training data used to train transformer-based LMs. The posterior probabilities of the two types of grammars are a way to measure which grammar better balances the trade-off between the goodness of fit and simplicity. Our aim is to check whether the trained transformer LM exhibits generalization consistent with choosing the simpler (i.e., larger posterior) grammar. We evaluate this by comparing the negative log-likelihood (NLL) assigned by the transformer LM to the test sets corresponding to the two generalizations. E.g. for the transformer model trained using data derived from $\mathcal{D}_{\mathrm{train-L}}$, we evaluate its NLL on the generalization test sets derived from the two grammars $\mathcal{D}_{\mathrm{test-L}}^{\texttt{Hier}}$ and $\mathcal{D}_{\mathrm{test-L}}^{\texttt{Lin}}$, and check if it assigns a lower NLL to the test data coming from the simpler grammar. Note that we compute the average NLL overall all examples in a test set for the final value. For a more intuitive metric, we also compute the main-verb accuracy – fraction of test examples for which the verb predicted by the model agrees with the hierarchically associated noun in the sentence. A main-verb accuracy of 1 indicates that the model’s generalization is consistent with the hierarchical rule and an accuracy of 0 when it is consistent with the linear rule.

The $\log$-probabilities for all the hand-constructed grammars on the two datasets is provided in Table 2. On both datasets, the one-state grammar gets the highest prior, which is expected as it is the simplest grammar that we study. However, the one-state grammar also fits the data the worst which is indicated by the lowest $\log$-likelihood (for both datasets). The flat grammars fit both the datasets the best and have the highest $\log$-likelihood, which is also expected since a flat grammar memorizes the training data. But it can come at a cost of increased complexity, especially when the training data is diverse; and so the flat grammar has the lowest $\log$-prior on the full dataset.

For the high diversity dataset $\mathcal{D}_{\mathrm{train-L}}$, we observe that the CFG best balances the tradeoff between the simplicity and goodness of fit, obtaining the highest posterior. This shows why it would be more beneficial to model this dataset using a hierarchical phrase structured grammar than a linear grammar. However, when we consider the low-diversity dataset $\mathcal{D}_{\mathrm{train-S}}$, while the CFG still obtains a better posterior than the regular grammar, it is the one-state grammar obtains the highest posterior out of all the grammars. This is consistent with the findings of Perfors et al. (2011), who found that for small corpora, one-state grammars often obtain higher posteriors than the context-free and regular grammars. In such cases, learning the distribution of syntactic category sequences, without abstract nonterminals, wins out on the Bayesian criterion.

We obtain consistent findings with some subtle differences for the grammars minimized using the Bayesian model merging algorithm, which we detail in §A.3.

Note that choosing the prior is subjective and can influence these results. Hence, to be extra careful, we conduct a sensitivity analysis by varying the values of the geometric distribution parameter $p$. We experiment with $p\in\{0.01,0.1,0.2,0.3,\cdots,0.9,0.99\}$ for the probability distribution on the nonterminals ($p(|V|)$) and number of productions ($p(P_{k})$), and obtain findings consistent with those in Table 2 (see Figure 14(c) in Appendix). We also experiment with having different values of $p$ parameter for $p(|V|)$ and $p(P_{k})$, and try out 49 combinations ($\{0.01,0.1,0.3,0.5,0.7,0.9,0.99\}\times\{0.01,0.1,0.3,0.5,0.7,0.9,0.99\}$) . For each of these combinations, we find that for $\mathcal{D}_{\mathrm{train-S}}$ case, consistent with Table 2 the CFG-S always obtain a lower posterior compared to the One-State grammar. Similarly for the CFG-L and Reg-L, the findings are also consistent across all 49 combinations i.e. CFG-L always obtain a higher posterior than Reg-L.

We train the transformer-based LMs on the two datasets ($\mathcal{D}_{\mathrm{train-L}},\mathcal{D}_{\mathrm{train-S}}$) and evaluate their generalization based on the $\mathcal{D}_{\mathrm{test}}^{\texttt{Hier}}$ and $\mathcal{D}_{\mathrm{test}}^{\texttt{Lin}}$ test sets. Note that for both datasets, 50k training examples are used. Recall that the two training datasets differ in their diversity (120 types in $\mathcal{D}_{\mathrm{train-L}}$ vs. 12 in $\mathcal{D}_{\mathrm{train-S}}$). We use the same experimental setup as discussed in §3.2. In Figure 5(a), we see for the models trained on the low-diversity dataset $\mathcal{D}_{\mathrm{train-S}}$ that the model obtains similar negative log-likelihood values on both test sets, implying that the model has no preference for generalizing according to the linear rule or the hierarchical rule. For this dataset, neither the CFG nor the regular grammar were optimal in terms of the posterior probabilities, so we observe that the transformer’s learning behavior is consistent with the “idealized” setup above. For the models trained on the $\mathcal{D}_{\mathrm{train-L}}$ dataset, however, we see that the model learns to generalize hierarchically, with the NLL on the $\mathcal{D}_{\mathrm{test}}^{\texttt{Hier}}$ test set being significantly lower than that on the $\mathcal{D}_{\mathrm{test}}^{\texttt{Lin}}$ test set.

Besides NLL, we also compute the main-verb accuracy, where we check whether the model assigns a higher probability to the main verb agreeing with the correct inflection form than the incorrect one. As we can see in Figure 5(b), the model trained on the $\mathcal{D}_{\mathrm{train-L}}$ dataset obtains close to 100% accuracy when evaluated on the $\mathcal{D}_{\mathrm{test}}^{\texttt{Hier}}$ test set. However, the model trained on the $\mathcal{D}_{\mathrm{train-S}}$ dataset obtains close to 50% generalization accuracy, again showing no preference for a hierarchical or linear rule (the latter would lead to 0% accuracy). We also verify that these results are not just a by-product of the choice of hyperparameters, and train transformer models with different layers (2, 4, 6, 8, 12), on the $\mathcal{D}_{\mathrm{train-S}}$ dataset, and in none of the cases did we observe the models exhibiting preference for hierarchical generalization (see results in Appendix Figure 18).

We also consider a stricter metric: all-verb generalization accuracy which is obtained by checking whether all predicted verbs (and not just the main verb) in the sentence have the correct inflection. The reason for considering this metric is that, for the agreement task, 100% main-verb generalization accuracy can also be obtained without learning the hierarchical rule and simply agreeing with the first noun in the sentence. Note that the all-verb accuracy is computed by feeding prefixes preceding each verb in the sentence and obtaining the model’s predictions. We provide the results with all-verb generalization accuracy in Figure 5(c), where we show that a baseline that always selects the verb to agree with the first noun in the sentence obtains 33% accuracy according to this metric, but as we can see transformers perform substantially better than this baseline, indicating that they do not trivially learn this heuristic to obtain high main-verb accuracy.

Results from this study indicate that when transformers-based LMs exhibit hierarchical generalization, despite being trained on ambiguous data, under the tested conditions, the hierarchical grammar not only fits the data well but is also simpler compared to the regular grammar with linear agreements. For the cases where this condition does not hold, we observe that the trained transformers exhibit no preference for hierarchical generalization.

First, it may be possible to find better grammars, either context-free or regular, in terms of posterior, on our two datasets. While local search using Bayesian model merging aims to find strong candidates, the possibility of better grammars cannot be eliminated as searching the space of all possible grammars is intractable. Secondly, our study treats transformers as black boxes and we do not answer whether or how these models implement hierarchical structure internally. The results from Murty et al. (2023a), which show that the representations of the transformer models become more and more tree-structured over the course of training, do provide some relevant evidence. Future work might focus on obtaining mechanistic explanations of this phenomenon.