Reason-able embeddings: Learning concept embeddings with a transferable neural reasoner

2.1.Deep learning

While machine learning is ubiquitous in research these days, we give a very brief introduction to ensure there is no confusion in terms. In this, we follow [31], and the reader is welcome to the book for a more extended introduction.

Classification is a task where a learning algorithm is supposed to produce a function, called a classifier, mapping an object to a label from a discrete, finite set of labels. A special case, used in this paper, is binary classification, where only two mutually-exclusive labels are considered: 0 (negative) and 1 (positive). Objects are usually represented by vectors of numerical features, albeit different representations are possible. By convention, features are usually denoted by x, and predicted labels are denoted by yˆ.

Binary classification is frequently solved by constructing a function predicting the probability of an object to belong to the positive class, and then thresholding on the probability to obtain the label. Throughout this work, we employ this approach, writing P(x) to denote the probability that the object represented by x should be labeled with the positive label.

A classifier usually contains a number of parameters, which are automatically adjusted during the training, a process aiming to optimize the value of some metric on a training set. The training set consists of examples, each example being an object assigned a true label that the classifier is supposed to predict. By convention, the true labels are usually denoted as y. The details of the optimization process can vary wildly, but in the context of this work, as the metric we use binary cross-entropy and employ a variant of gradient descent optimization algorithm called AdamW [53] to minimize it.

For gradient descent algorithms, it is established to use a mini-batch training, where the training set is randomly split into small, disjoint subsets called mini-batches, and each mini-batch is used to update the parameters of the model separately. A complete pass over all the mini-batches from a training set is called an epoch. In some special cases, only a subset of parameters is optimized at a given time, while the remaining are called frozen and kept constant.

Due to the stochastic nature of training, too aggressively optimizing the metric can lead to overfitting, a phenomenon in which the classifier labels the training examples very well, but fares poorly on new, unseen data. To evaluate the performance of a classifier on unseen data, a separate set, called test set, is maintained. It is assumed that both sets are independent and identically distributed (the i.i.d. assumption), i.e., the examples in them are drawn independently of each other from the same probability distribution. Usually, for the evaluation, different metrics are used than during training. The details of the metrics used in this paper are given in Section 6.3.

It is frequently the case that a learning algorithm has some configuration options that cannot be automatically adjusted using the training set, e.g., because that would immediately lead to overfitting. These options are called hyperparameters, and are adjusted by observing the value of a metric of choice on yet another set, the validation set. It follows the same i.i.d. assumption as the two other sets.

A feed-forward neural network is a sequence of linear transformations and activation functions, where each transformation is followed by an activation function. By convention, each pair: a linear transformation and an activation function is called a layer. The linear transformation is of form Ax+b, where A is a matrix of the type n×m, b is a vector of the type n, and x is a vector of the type m. A and b are parameters, called weights, while x is an input. By convention, m is called the number of neurons of the layer. Throughout the paper, we denote a single layer with Lmf, where m is the number of neurons and f is the activation function: Lmf(x)=f(Ax+b). In a special case where there is no activation function in a layer, we replace f with ·. In this convention, a feed-forward neural network of two layers, the first consisting of 16 neurons and employing the activation function σ and the second of 1 neuron and employing no activation function, can be represented as: L1·(L16σ(x)). Traditionally the last layer is called the output layer, and the remaining layers – hidden layers.

Over the years, many activation functions were considered. Throughout the paper we use two of them: the sigmoid σ, given in Equation (1), and ELU in Equation (2) [15].

Finally, by an embedding we understand a real-valued vector representing some concept from a non-numeric set, such that the size of the vector is much smaller than the cardinality of the set, and that the dimensions are not directly interpretable, i.e., the vector comes from a latent space.

2.2.Description logics

Description logics (DLs) are a family of logics, that are widely used as a formal way to represent knowledge in the form of knowledge bases (KBs). An advantageous property of DLs is that reasoning in most of them is decidable, since they are fragments of first-order predicate logic [1]. The formalism of DLs has been used as a basis for the Web Ontology Language (OWL) standard, which as part of the Semantic Web is responsible for describing the semantics, or meaning, of data on the Internet [59]. DLs and the foundational technologies of the Semantic Web are considered mature, and are used both in research and the industry [36]. An illustrative example of the potential applications of DLs is SNOMED CT – a knowledge base that is considered to be the most comprehensive multilingual clinical healthcare terminology in the world [5].

As mentioned, DLs are a family of logics, which means that there are many DLs of varying expressiveness. Since in our work we focus on the ALC DL (abbreviated from attributive language with complements), we also introduce DLs from the perspective of ALC. There exist DLs that are simpler than ALC, for example EL, and there are also vastly more expressive DLs like SHOIN(D) and SROIQ(D), which are the basis of OWL DL and OWL 2 DL respectively. We do not deal with EL or SROIQ(D), but we do discuss a procedure for transforming SHOIN(D) knowledge bases to ALC knowledge bases in Section 6.4.2, by removing parts that ALC does not support.

For readers that are unfamiliar with DLs we begin with an intuitive overview. In later sections we describe the notation we use, and define the syntax and semantics of the ALC DL. The introduction to DLs is based on [78].

3.1.Knowledge base embeddings

One way to bridge the neural and symbolic paradigms is to represent symbolic knowledge in terms of vectors in a high-dimensional real vector space. Such vectors can then be used as additional inputs to machine learning models based on neural networks, to improve their performance by allowing them to use an approximation of expert knowledge [87]. A good method of learning embeddings from symbolic knowledge should ideally leverage the structural information present in relations between abstract concepts, and should not try to learn embeddings for abstract concepts with the aid of word embeddings, due to the ambiguity of language, its limited abstraction, and other problems [11].

One of the first embeddings methods tailored to KBs to get traction were NTN proposed by Socher et al. [84], TransE proposed by Bordes et al. [7,8], later unified into a single framework by Yang et al. [97], as both leverage linear and bilinear mappings. The notion of modeling relations as operations in some vector space was further extended by Wang et al. with TransH [94], Lin et al. with TransR [51] and PTransE [50], Sun et al. with RotatE [85], Zhang et al. with HakE [98], Wang et al. with InterHT [90], or Zhou et al. with Path-RotatE [99].

A separate category of embeddings was established by ConvE, employing convolutional neural networks [20]. This line of research was continued by Nguyen et al. with ConvKB [62], Jiang et al. with ConvR [42], Demir et al. with ConEx [19]. Over the time even more complex neural architectures were employed, e.g., capsule networks in CapsE by Nguyen et al. [63], recurrent skipping networks by Guo et al. [33], graph convolutional networks in GCN-Align by Wang et al. [93] or R-GCN by Schlichtkrull et al. [82], recurrent transformers by Werner et al. [95]

Yet another approach was taken in RESCAL by Nickel et al. where tensor-based techniques were used [65]. Similar approaches were taken in TOAST by Jachnik et al. [39], TATEC by García-Durán et al. [28], DistMult by Yang et al. [97], HolE by Nickel et al. [64], ComplEx by Trouillon et al. [88], or ANALOGY by Liu et al. [52].

In the context of the Semantic Web technologies, notable examples of embeddings are RDF2Vec by Ristoski et al. [75], OWL2Vec^* by Chen et al. [12], and TransOWL by d’Amato et al. [17].

Embeddings are used in various downstream tasks, such as disambiguation [80], ontology learning [71], handling concept drift in streams [13], question answering [54] or fact classification [43]. Over the years multiple survey papers were published on the topic of KB embeddings [10,14,16,91,92].

To the best of our knowledge, none of the proposed methods aims to capture the semantics as a topology of the embedding space, nor enables computing the embeddings of complex expressions from the embeddings of their parts.

3.2.Deep deductive reasoners

Compared to the work on embeddings, much less attention has been devoted to the notion of approximating or emulating the results of deductive reasoning with machine learning, and in particular deep learning [23].

Earlier works, employing traditional machine learning approaches, were mostly devoted to approximate a deductive reasoner in order to obtain the results faster than from a deductive reasoner. For example, Rizzo et al. [76,77] proposed terminological trees and forests to tackle this problem, while Paulheim and Stuckenschmidt concentrated on the problem of approximate reasoning with ABox [70].

One of the first works leveraging deep learning for deductive reasoning over ontologies were those of Makni and Hendler on using recurrent neural networks for RDFS reasoning [56], and of Hohenecker and Lukasiewicz on using recursive reasoning networks [37]. Eberhart et al. employed long short-term memory networks (LSTMs) for reasoning in the EL+ description logic [22,55]. Badreddine et al. introduced Logic Tensor Networks [2], which were then employed by Bianchi and Hitzler to perform deductive reasoning [6]. Ebrahimi et al. investigated the applicability of pointer networks to the task [24] and of memory networks [25]. Zhu et al. cast the problem of reasoning as a graph-to-graph transformation problem and applied it in the context of RDF [100].

Ebrahimi et al. in [24] introduced a few dimensions along which a reasoner can be classified: logic – which logical formalism is tackled; transfer – the capability of achieving good performance on a previously unknown KB; generative – the capability of generating new inferences, not only answering yes/no queries; scale – the size of considered KBs; performance – high if the system was able to cross the 70% threshold on the F1-score, low otherwise. Following these dimensions, the reasoner presented in this work tackles the ALC logic, is transferable, is not generative, is of low to moderate scale, and has high performance. To the best of our knowledge, this is the first transferable reasoner capable of tackling such a complex logic, albeit we must underline that the embeddings for a new KB must be trained under the supervision of the reasoner.

4.1.Reasoner head

In our work we chose the reasoner to be an entailment classifier for subsumption axioms in ALC – that is, given an axiom C⊑D, the reasoner head outputs the probability P(K⊧C⊑D) that the axiom is entailed by a given knowledge base K. Assume that hK is a function mapping ALC concepts in knowledge base K to embedding vectors in real vector space RNe, where Ne is the embedding dimension. The details of how this function is constructed are given in Section 4.2.

An interaction map for a pair of embeddings u, v, denoted IM(u,v), as defined by Equation (3), is a concatenation of u, v, and their outer product11 u⊗v. Before concatenation, the outer product is reshaped to a row vector. Our preliminary experimentation indicated that including the outer product in the interaction map is of utmost importance, as this helps the classifier to learn high-order correlations [3,34]. By extension, an interaction map for a pair of concepts C, D in knowledge base K is the interaction map for their embeddings: IMK(C,D)=IM(hK(C),hK(D)).

The classification output for an entailment query P(K⊧C⊑D)∈(0,1) (see Equation (4)) is defined as the output of the feedforward neural network NN⊑, which accepts the interaction map of C and D, and has a single output neuron with the sigmoid activation function σ and a single hidden layer with 16 neurons, which is followed by the ELU activation function [15].

4.2.Embedding layer

Recursive neural networks have been successfully used for encoding expression trees as fixed-size vectors, that could be used as inputs to machine learning models [30]. The recursively defined DL concepts also have a tree structure [49], so it is appropriate to use a recursive neural network as the embedding layer in our reasoner architecture. The key hypothesis that defines our reasoner is that for each KB, we can train an embedding layer that embeds concepts from that KB in an embedding space with a topology that makes it easy for the reasoner head to classify entailment. An embedding topology that is beneficial to entailment classification is formed by jointly training the reasoner head and embedding layers for multiple KBs, which forces the embedding head to generalize, and the embedding layers to output embeddings in the shared embedding space.

We note in passing that we distinguish between a recursive and a recurrent neural network: in a recursive neural network, different parts of the network are applied to the input multiple times, depending on a structured input, as with recursive grammars, where the same production rule can be applied again and again; in a recurrent neural network (e.g., GRU, LSTM) the output of the network is connected back to the network, as part of its input.

In general, concept embeddings are represented by vectors in real vector space RNe, where Ne is the embedding dimension. In Equations (5)–(11) we define the recursive function hK:ALC→RNe that maps ALC concepts for knowledge base K to embedding vectors in real vector space RNe. We begin with the embedding for each concept name Ai: a vector WK,Ai of dimension Ne, as given by Equation (5). The vector is optimized during training and it is KB-specific, since different KBs have a different number named concepts with different meanings.

The embeddings of the bottom and top concepts are stored by the vectors W⊥ and W⊤, respectively, as specified by Equation (6). Since the semantics of the top and bottom concepts are independent of any specific KB, the vectors are shared between all KBs.

To obtain the embedding for the complement of a given concept, one first recursively obtains the embedding of that concept, and then passes it as an input to the complement constructor network NN¬, consisting of a single layer of Ne neurons without the activation function, as defined in Equation (7).

We also define in Equation (8) the embedding of a doubly negated concept as the embedding of that concept, by the double negation elimination rule, which speeds the construction of embeddings.

To obtain the embedding for an intersection of concepts, one first recursively obtains the embeddings of the two concepts, computes their interaction map, and passes it as the input to the intersection constructor network NN⊓. It is a single layer of Ne neurons without the activation function, as specified by Equation (9), that maps the interaction map of two concept embeddings, to the embedding of their intersection. Its input is a vector of the size of the interaction map 2Ne+Ne2, and the output is a vector of size Ne.

Fig. 2.

An example interpretation of a knowledge base with role assertions is shown on the left side. The individuals Fido, Salem, and Rocky are elements of the set AnimalI. Individuals Alice, Bob, and Charlie are elements of set (∃owns.Animal)I. An illustration of transformations from concept embeddings to existential restriction embeddings is shown on the right side. The embedding of ∃owns.C may be obtained by applying NNK,owns to the embedding of concept C.

To obtain the embedding for an existential restriction, one first obtains the embedding of the given concept, and passes it to the existential restriction constructor network, defined as NNK,Ri=LNe·, i.e., consisting of a single layer of Ne neurons without the activation function. To understand the intuition behind this choice, consider the following example. On the left side of Fig. 2 we show an interpretation of a KB with a set of human individuals, and a set of individuals that are instances of concept Animal. Some individuals own an animal, which is described with role assertions owns(Alice,Fido), owns(Bob,Fido), owns(Charlie,Rocky) (shown as arrows). Alice, Bob and Charlie may be described as instances of ∃owns.Animal. Notice that if we reversed all owns arrows, then an owns arrow would always start in set Animal, and end in set ∃owns.Animal, as by definition (∃R.C)I={x∈ΔI|∃y∈ΔI((x,y)∈RI∧y∈CI)}, all elements of (∃R.C)I are related to some element of CI. So in general, if we visualize role assertions as arrows, for any interpretation I, if we reverse role Ri arrows, they always begin in set CI and end in set (∃Ri.C)I. Now consider the right side of the figure. Our embedding layer represents concepts as points in the embedding space RNe and does not support individuals at all, so we “squish” the sets ∃owns.Animal and Animal into points in RNe, but we keep the (reversed) arrows, since their start and end points always occur inside the sets. Thus, we think that it is appropriate to model the existential restriction constructor as a function mapping concept embeddings hK(C) to embeddings of the existential restriction hK(∃Ri.C). There is a separate function NNK,Ri for each role, and functions NNK,Ri need to be learned per knowledge base K, because different KBs have different numbers of roles, and roles may have different meanings, so there is no obvious way of sharing existential restriction constructors between KBs. The formalization is given in Equation (10).

In deep learning, activation functions must be used after each hidden layer so that one does not compose multiple linear transformations, which are equivalent to a single linear transformation. Note that even though our constructor networks do not have an activation function, the result of the hK function is not simply a composition of multiple linear transformations, because at every recursive step the constructor networks (i.e. linear transformations) are given different inputs. If one would like to make the constructor networks deep, i.e., add more hidden layers, then one must add an activation function like ELU after each layer, except the last one.

To extend hK to concept unions and universal restrictions, we use the de Morgan’s laws [78], arriving at the formulas given in Equation (11).

4.3.Relaxed architecture

An important property of the embedding layer, that we introduced in the previous section, is that the weights of the neural networks NN¬ and NN⊓ do not depend on a specific knowledge base. Since the semantics of concept complement and intersection constructors do not change depending on a KB, we think that their neural equivalents should also not change. We call a reasoner, where NN¬ and NN⊓ network weights are stored in the reasoner head, the restricted reasoner. Unless stated otherwise, we use restricted reasoners.

While sharing constructors mimics the logic, it may not be an appropriate assumption in a deep learning model and thus a hindrance to learn good embeddings. To verify whether this is the case, we consider a variant of the reasoner, that allows the embedding layer to learn KB-specific concept constructor networks NNK,¬ and NNK,⊓, and a KB-specific embedding of the bottom concept WK,⊥ and the top concept WK,⊤. We call this variant the relaxed reasoner. All changes to the function hK in the relaxed reasoner are marked (underlined) below.

In the relaxed architecture, the only weights shared between different KBs are the ones learned by the classifier NN⊑, which we expected would allow the embedding layers to learn more fitting embeddings, at the cost of limiting generalization and opportunities for transfer learning.

4.4.Training procedure

The training procedure for our reasoner consists of two steps. In the first step, we train both the reasoner head and embedding layers for as many diverse KBs as possible. This results in a reasoner head that learned to classify whether subsumption axioms hold in any KB, given that an appropriate embedding layer is provided. By appropriate embedding layer we mean a layer that learned to embed KB-specific concepts in a space that minimizes the classifier loss. We simply call this step training the reasoner head, and we consider the data used in this step as the training set. Note that, as a result of training the reasoner, we obtain trained embedding layers for KBs in the training set. If obtaining the trained embedding layers was the goal, then the next step is not necessary.

In the second step, we freeze the reasoner head and train the embedding layers for KBs that were not seen in the first step. This results in embedding layers that can embed concepts in a space in which the reasoner is good at classification. We think that if our reasoner can accurately classify whether subsumption axioms are entailed by a KB, then the embeddings used as inputs to the classifier NN⊑ faithfully capture the semantics of the KB, even though we did not train the reasoner head in the second step. We call this step training the embedding layers or testing the reasoner head, and we consider the data set used in this step as the test set.

6.1.Goals

Although the design of the proposed approach mimics the ALC logic and is well-grounded in research on deep learning architectures, ultimately its usability and properties must be established experimentally. To this end, we formulate four research questions and design appropriate experiments to answer them.

The main hypothesis of our research is that using a transferable reasoner head forces the embeddings to reflect the semantics of the underlying KBs in a shared way. To address it, we pose the following research question RQ1: Can the reasoner induce a useful topology that it can subsequently exploit, or are the embeddings simply overfitting the KBs? We answer RQ1 in Experiment 1, described in Section 6.5, where we compare the classification metrics of a trained reasoner and a random reasoner in previously unseen ontologies, and show that the trained reasoner indeed brings something to the table.

In Section 4.3, we constructed a relaxed variant of the reasoner, hypothesising that the restricted variant may be too restrictive and impede the learning process. From this, we formulate RQ2: Can concept constructors be shared between KBs, or must they be KB-specific? To answer it, we designed Experiment 2, described in Section 6.6. In there, we follow the same procedure as in Experiment 1, but using the relaxed reasoner. We then compare the results obtained by the restricted reasoner in Experiment 1, and by the relaxed reasoner, and show that the difference is negligible.

The overarching goal of this work is to present a method to train embeddings for ALC KBs including the semantics of the concepts, and thus we pose RQ3: Can the presented approach learn good embeddings for concepts from a real-world ontology? We propose Experiment 3, described in Section 6.7, where we use the pizza ontology as the real-world ontology. We consider three different training schemes, measure the classification metrics, and visually analyze the obtained embeddings, showing that the reasoner head from Experiment 1 fares only slightly worse than a reasoner head overfitted to the ontology.

We extend RQ1 and RQ3 to a larger set of real-world ontologies, formulating RQ4: Is transfer learning with the presented approach viable for a variety of real-world ontologies? In Experiment 4, described in Section 6.8, we follow the methodology established in Experiment 3 with 6 different ontologies of varying sizes and complexity and show that on average the results of using a pre-trained reasoner head are as good as training a reasoner head from scratch on the considered ontologies.

Finally, we tackle the problem of identifying what is hard for the reasoner by posing RQ5 Are there types of queries that are significantly harder for the reasoner than others?. We briefly address this question in Experiment 5, however, without reaching a satisfactory answer.

6.2.Setup

We implemented the proposed approach in Python 3.9.7 and Cython [4] (a superset of Python that compiles to C or C++), and used PyTorch 1.10.1 to implement the neural reasoner [68]. The source code is available at https://github.com/maxadamski/reasonable-embeddings. We ran the experiments on a computer with an Intel Core i5-4670K CPU, 32 GB of DDR3 RAM, and no dedicated GPU, running Void Linux with kernel 5.15.

We use pseudo-randomly generated numbers to create our data sets and in training. For data generation we use the NumPy implementation of the Permuted Congruential Generator [66]. The internal PyTorch pseudo-random number generator is used during training. To ensure reproducibility of our experiments, we set the initial states of the pseudo-random number generators to a known initial value.

During training, our reasoner uses inferences made by a semantic reasoner as the expected class in classification. The Semantic Web community traditionally uses Java as the language of choice, a language which does not interface well with Python. In particular, using either HermiT [61] or Pellet [83], state-of-the-art semantic reasoners, required executing the reasoners as separate Java Virtual Machine (JVM) processes, a process for every inference. This introduced unacceptable overhead, which we alleviated by employing FaCT++ [89], a semantic reasoner implemented in C++, which makes the task of interfacing with Python much easier. We improved a Python interface for FaCT++ by wrobell,22 extending it with missing concept constructors, and addressing performance issues.

In the experiments dealing with OWL ontologies, initially we used Owlready2 [47] to parse OWL ontologies in the RDF/XML format. Eventually, to improve performance and avoid some problems with the parser, we switched to converting the ontologies to the OWL functional-style syntax [69] using the ROBOT command-line tool [40], and parsing them using a custom, high-performance parser implemented in Cython.

Both our parser and the extended version of the Python interface for FaCT++ are available at https://github.com/maxadamski/reasonable-embeddings/tree/main/src/simplefact, together with compilation instructions in the root of the repository.

6.3.Evaluation metrics

As the quality of the learned embeddings cannot be measured directly, we perform so-called extrinsic evaluation by computing classification metrics for a trained reasoner. We stipulate that if the classification metrics indicate good performance, then the embeddings learned by the embedding layer must capture the semantics of a given KB well. If that was not the case and the embeddings did not encode useful information for inference in the ALC DL, then the classifier should not be able to reliably classify entailment of axioms.

For simplicity, in threshold-sensitive metrics we choose a threshold of 0.5, i.e, yˆ=1 if P(K⊧α)⩾0.5, and yˆ=0 otherwise. Given that y is the expected class, TP is the number of true positives, i.e., the number of queries where yˆ=y=1; TN is the number of true negatives, i.e., the number of queries where yˆ=y=0; FP is the number of false positives, i.e., the number of queries where yˆ=1 and y=0; and FN is the number of false negatives, i.e., the number of queries where yˆ=0 and y=1.

Our classifier learns binary classification, so we chose appropriate metrics [86]. Firstly, we include accuracy (Equation (16)) in the set of evaluation metrics. Because the data set that we generated for the experiments is slightly imbalanced we also compute precision (Equation (17)), recall (Equation (18)), and the F1-score (Equation (19)).

We also compute the values of the area under the ROC curve (AUC-ROC) and the area under the PR curve (AUC-PR), which are threshold-invariant metrics with a minimum value of 0 and a maximum value of 1. The ROC curve shows the performance of a classification model at all classification thresholds, by plotting the true positive rate (TPR; also called recall) against the false positive rate (FPR) (Equation (20)). Similarly, the PR curve shows the performance of a model at all thresholds by plotting precision against recall [79]. For both ROC and PR curves, a larger area under the curve suggests a better classifier [26,79].

In experiments leveraging multiple KBs, we compute all metrics separately for each query set, and then compute the average and standard deviation across KBs. We do this because the KBs in the test set may be unequally difficult to classify, so it is beneficial to measure the variance of the reasoner performance across different KBs.

6.4.Data sets

6.5.Experiment 1 – topology induction on the synthetic data set

In this experiment, we consider RQ1: Can the reasoner induce a useful topology that it can subsequently exploit, or are the embeddings simply overfitting the KBs? To answer it, we designed a simple test to check whether a trained reasoner head actually learns to reason in the ALC description logic, and high classification metric values are not just the effect of embedding layers overfitting to KBs.

We test our architecture by first training the reasoner head and embedding layers on the training set (as usual), which results in a skillful reasoner. Then we create a no-skill reasoner head, that is not trained, but only randomly initialized. For each of the two reasoner heads, we train embedding layers on the test set, but keep the reasoner head weights frozen. After training the embedding layers on the test set for a fixed number of epochs, if the classification metrics for the reasoner with the trained head are significantly greater than for the reasoner with the random head, then the trained reasoner head learned useful relations in the ALC embedding space that generalize to new KBs. In short, the reasoner head is transferable.

Conversely, if the differences between classification metrics of both reasoner heads are not significant, then the trained reasoner actually has little or no skill and the only skill in classification comes from the embedding layer, which would mean that the reasoner head is not transferable.

6.5.2.Evaluation of reasoning ability

We evaluate the reasoning ability of the restricted variant of our reasoner by monitoring the training and validation loss and AUC-ROC values during training. The only goal of this assessment is to verify that the reasoner can effectively learn to classify entailment in the training set, and does not suffer from underfitting. Training and validation losses steadily decreasing, and validation AUC-ROC increasing with each training epoch, suggests that the reasoner architecture is sufficient for learning to classify entailment in a given data set.

Fig. 3.

Training and test progress of the restricted reasoner. The reported training loss for each epoch is the average mini-batch loss in that epoch. We also show the standard deviation of the mini-batch loss for each epoch. In test progress, the trained reasoner is shown in red, while the random reasoner is shown in blue. The reasoner was not trained during epoch 0 of training and testing – during epoch 0 we only compute the initial loss and metric values. Dashed curves show the AUC-ROC for each KB in the test set, while the thicker blue and red curves are the averaged AUC-ROC across KBs in the test set. Note that the random reasoner loss actually decreases, but at a very slow rate.

As shown in Fig. 3, the training loss decreases and validation AUC-ROC increases for the entirety of the training, with smaller gains after epoch 10. Furthermore, the validation loss does not start increasing in later epochs, which suggests that the restricted reasoner variant is not prone to overfitting.

6.6.Experiment 2 – comparing restricted and relaxed reasoners

In this experiment we answer RQ2: Can concept constructors be shared between KBs, or must they be KB-specific? We followed the same protocol as in Experiment 1, but used the relaxed reasoner instead of the restricted reasoner.

6.6.2.Evaluation of knowledge transfer

Fig. 4.

Training and test progress of the relaxed reasoner. The reported training loss for each epoch is the average mini-batch loss in that epoch. We also show the standard deviation of the mini-batch loss for each epoch. In test progress, the trained reasoner is shown in red, while the random reasoner is shown in blue. The reasoner was not trained during epoch 0 of training and testing – during epoch 0 we only compute the initial loss and metric values. Dashed lines show the AUC-ROC for each KB in the test set, while the thicker blue and red curves are the averaged AUC-ROC across KBs in the test set.

After training the relaxed reasoner, we froze the reasoner head, and trained embedding layers on the test set. We also trained embedding layers in conjunction with a randomly initialized reasoner head. The test progress is shown on the right side of Fig. 4. At the beginning, the test loss for the trained reasoner head was higher than the test loss for the randomly initialized reasoner head. The test loss decreased quickly for the reasoner with the trained head, while the loss for the random head decreased very slowly. For the reasoner with the trained head, the test AUC-ROC quickly increased to about 0.7 after epoch 2, and approached 1 after the last epoch. The average test AUC-ROC for the random head slowly increased from around 0.5 in the beginning to around 0.8 after the last epoch.

Overall, training embedding layers for the relaxed reasoner with the trained head was much faster, than for the reasoner with the randomly initialized head. Moreover, the trained head allowed the reasoner to achieve an average AUC-ROC close to 1 on the test set after 10 epochs, while the reasoner with the random head only achieved an average AUC-ROC of around 0.8.

We report the final metrics of training on the test data in Table 4. For the relaxed architecture, the trained model is strictly better than the randomly initialized one, as all metric values are higher for the former. Moreover, the trained model has lower variance of metrics across KBs in the test set, than the random model. Similarly as for the restricted reasoner, the reasoner with trained head outperforms the reasoner with random head by a fair margin.

6.7.Experiment 3 – visualizing concept embeddings from a real-world ontology

The previous experiments showed that our reasoner is capable of learning good concept embeddings for synthetic KBs, so the next step is to answer RQ3: Can the presented approach learn good embeddings for concepts from a real-world ontology? In this experiment we do that by training the neural reasoner to classify entailment, given randomly generated queries about the pizza ontology as the data set. We repeat this experiment three times. In the first run, we let the reasoner learn without any pre-training. In the second run, we initialize the reasoner head with weights of the restricted reasoner, that we trained in Experiment 1, then freeze it, and only allow the embedding layer to learn. In the third run, we randomly initialize the reasoner head, and also freeze it to only allow the embedding layer to learn KB-specific embedding.

6.7.5.Embedding analysis

In addition to evaluating the reasoning and transfer ability of our reasoner, which showed that it can successfully learn to classify entailment in a real-world KB, we visualize the learned embeddings. A 2D visualization that tries to preserve the distances between concepts in the high-dimensional embedding space enables visual assessment of the quality of the learned embeddings. We think that if semantically similar concepts are placed closer to each other than to dissimilar concepts, then the learned embeddings capture the semantics of a given KB.

We set the concept embedding dimension to Ne=10, so to visualize the embeddings we first needed to reduce their dimensionality. We used Uniform Manifold Approximation and Projection for Dimension Reduction (UMAP),1010 because it is a nonlinear dimensionality reduction algorithm with high performance and interpretable hyperparameters [60], that we set as follows: n_neighbors to 50 to emphasize the global structure of the data, and set min_dist by starting at a large value and decreasing it until clusters started to separate and the visualization became readable, which occurred when min_dist was set to 0.3.

The UMAP visualizations of the embeddings learned in this experiment, are shown in Fig. 5. The visualization of embeddings learned by the unfrozen reasoner suggests, that the embeddings capture the semantics of the pizza ontology well. As expected, the unsatisfiable CheesyVegetableTopping is close to ⊥, and the general DomainThing and Food concepts are close to ⊤. Spiciness and pizza bases form their own clusters. Pizzas and toppings are separated, with vegetarian pizzas forming one cluster, and non-vegetarian pizzas forming another cluster together with spicy pizzas. Cheesy toppings, vegetable toppings, spicy toppings, and pepper toppings are also close to related concepts.

The embeddings learned by the reasoner with the pre-trained head look a bit worse than those of the unfrozen reasoner. The pizzas and the toppings are separated, but inside of the topping and pizza clusters the embeddings are not as well organized.

The embeddings learned by the reasoner with the randomly initialized head do not look good when visualized. A few concept names form clusters that make sense, but most look like they are randomly scattered. General concepts are close to the top concept, which is good, but the unsatisfiable CheesyVegetableTopping is far away from the bottom concept.

In general, we think that the embeddings for the unfrozen reasoner are the best, the embeddings for the reasoner with the frozen pre-trained head are good, and the embeddings for the reasoner with the frozen randomly initialized head are not good at all. Again, the results of the visual assessment of the learned embeddings are consistent with the classification metrics of the reasoners.

Overall, the presented results indicate that the answer to RQ3 is affirmative, and that the presented approach can learn embeddings of good quality.

Fig. 5.

UMAP visualization of the learned concept embeddings. We replaced “Topping” in concept names with “T” to improve readability. We use different shapes, sizes, and colors of markers as follows: by default concepts are square and gray; the top concept, bottom concept and concept expressions are cyan; toppings diamond-shaped, and pizzas are round; vegetarian pizzas and toppings are light green, except vegetable toppings, which are dark green; seafood toppings are pink; non-vegetarian pizzas and meat toppings are dark red; cheesy pizzas, and cheese toppings are respectively marked with a yellow disk or yellow diamond inside; pepper toppings are marked with an orange diamond inside; spicy things are marked with a red “x” inside; spiciness levels are light red.

6.8.Experiment 4 – learning concept embeddings in different real-world ontologies

In Experiment 3 we found that a reasoner with a frozen pre-trained head achieves similar classification metrics to a reasoner trained from scratch. To see whether this result holds for a more diverse set of ontologies we posed RQ4: Is transfer learning with the presented approach viable for a variety of real-world ontologies?

6.9.Experiment 5 – what is difficult for the reasoner?

One can wonder what types of axioms are particularly hard for the reasoner. To answer this, we started with the reasoner and the test set from Experiment 1. Then, for each query in the test set, we computed the following features:

Currently, we do not have a decisive answer to RQ5. There do not seem to be any obvious indicators of hardness for a query. We stipulate it might be possible to detect more complex patterns by using, e.g., an adversarial attack [32], or frequent pattern mining [48,72]. However, we deem this to be out of the scope of this work.