Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19)

Dynamic Hypergraph Neural Networks

Jianwen Jiang1,2 , Yuxuan Wei1,2 , Yifan Feng3 , Jingxuan Cao1,2 and Yue Gao1,2∗
1
Beijing National Research Center for Information Science and Technology(BNRist)
2
KLISS, School of Software, Tsinghua University, Beijing, China
3
School of Information Science and Engineering, Xiamen University, Xiamen, China
{jjw17, weiyx15}@mails.tsinghua.edu.cn, [email protected], [email protected],
[email protected]

Abstract Embedding on the Embedding on the

𝒗𝟎 𝒍-th layer 𝒍+1-th layer
In recent years, graph/hypergraph-based deep 𝒗𝟏 𝒗𝟎
𝒆𝒍 𝒆𝒍𝒅𝒊𝒔𝒂𝒑𝒑𝒆𝒂𝒓
learning methods have attracted much attention Evolution 𝒗𝟔 𝒗𝟏
from researchers. These deep learning methods
take graph/hypergraph structure as prior knowl- 𝒗𝟔 𝒗𝟐 𝒗𝟐
edge in the model. However, hidden and im- 𝒗𝟓
𝒗𝟓 𝒗𝟑
portant relations are not directly represented in 𝒗𝟑
𝒗𝟒 𝒗𝟒 𝒍+𝟏
the inherent structure. To tackle this issue, we 𝒆
propose a dynamic hypergraph neural networks
framework (DHGNN), which is composed of the Figure 1: Dynamically constructed hyperedges. When embedding
stacked layers of two modules: dynamic hyper- evolves from the l-th layer to the l + 1-th layer, hyperedge el =
graph construction (DHG) and hypergrpah convo- {v0 , v1 , v2 , v6 } disappears while hyperedge el+1 = {v2 , v3 , v4 , v5 }
lution (HGC). Considering initially constructed hy- comes into existence.
pergraph is probably not a suitable representation
for data, the DHG module dynamically updates hy-
degree of hyperedges is restricted to 2, a hypergraph is de-
pergraph structure on each layer. Then hypergraph
generated to a simple graph, indicating that simple graph is a
convolution is introduced to encode high-order data
subset of the hypergraph.
relations in a hypergraph structure. The HGC mod-
ule includes two phases: vertex convolution and Recently graph/hypergraph-based deep learning methods
hyperedge convolution, which are designed to ag- have received more and more attention from researchers. In-
gregate feature among vertices and hyperedges, re- spired by convolutional neural network (CNN) [Krizhevsky
spectively. We have evaluated our method on stan- et al., 2012] in computer vision, researchers have designed
dard datasets, the Cora citation network and Mi- graph-based neural networks for semi-supervised learning,
croblog dataset. Our method outperforms state-of- like GCN [Kipf and Welling, 2017] and GAT [Veličković
the-art methods. More experiments are conducted et al., 2018]. Furthermore, HGNN [Feng et al., 2018] is
to demonstrate the effectiveness and robustness of the first hypergraph neural network model. In a neural
our method to diverse data distributions. network model, feature embedding generated from deeper
layer of the network carries higher-order relations that ini-
tial structure fails to capture. The major drawback of exist-
1 Introduction ing graph/hypergraph-based neural networks is that they only
employ the initial graph/hypergraph structures while neglect
Graphs are widely used to model pair-wise relations includ-
the dynamic modifications of such structures from adjusted
ing paper citations, personal contacts and protein-protein in-
feature embedding.
teractions. However, besides pair-wise relations, there exists
a large number of non-pair-wise relations that simple graphs Dynamic hypergraph structure learning (DHSL) [Zhang
are unable to model, for example, the communities in social et al., 2018] has been proposed to deal with this problem.
networks and the clusters in feature embeddings. Hypergraph DHSL uses raw input data to optimize hypergraph structure
is a generalized structure for relation modeling. A hypergraph iteratively. Nonetheless, DHSL only updates the hypergraph
is composed of a vertex set and a hyperedge set, where a hy- structure on initial feature embedding, thus failing to exploit
peredge contains a flexible number of vertices. Therefore, high-order relations among features. Also, the iterative op-
hyperedges are able to model non-pair-wise relations men- timization in DHSL suffers from expensive cost in time and
tioned above. The number of vertices a hyperedge contains space.
is defined as the degree of the hyperedge. Especially, if the To tackle these issues, we propose a dynamic hypergraph
neural networks (DHGNN) framework, which is stacked lay-
∗
Corresponding author. ers of dynamic hypergraph construction (DHG) module and

2635
 Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19)

Hypergraph Construction Hypergraph Convolution New feature

embedding

Vertex

…
Conv
Hyperedge Hyperedge

…
…
Conv

Centroid
Vertex

…
Vertex
Conv
Feature
Vertex Hyperedge
Hyperedge
Features Features

Figure 2: DHGNN framework. The first frame describes the hypergraph construction process on centroid vertex (the star) and its neighbors.
For instance, two hyperedges are generated from two clusters (dashed ellipses). In the second frame, features of vertices contained in a
hyperedge are aggregated to hyperedge feature through vertex convolution and features of adjacent hyperedges are aggregated to centroid
vertex feature through hyperedge convolutoin. After performing such operations for all vertices on current layer feature embedding, we obtain
the new feature embedding where new hypergraph structure will be constructed, as is shown in the third frame.

hypergraph convolution (HGC) module. In DHG module, we data distributions. On social media sentiment prediction,
utilize k-NN method and k-means clustering method to up- we observe performance improvement against state-of-
date hypergraph structure based on local and global features the-art methods.
respectively during a single inference process. Furthermore, The rest of the paper is organized as follows. Section 2 in-
we propose a hypergraph convolution method in HGC mod- troduces related work in graph-based deep learning and hy-
ule by a stack of vertex convolution and hyperedge convo- pergraph learning. Section 3 explains the proposed dynamic
lution. For vertex convolution, we use a transform matrix hypergraph neural networks method. Applications and exper-
to permute and weight vertices in a hyperedge; for hyper- imental results are presented in Section 4. Finally, we draw
edge convolution, we utilize attention mechanism to aggre- conclusions in Section 5.
gate adjacent hyperedge features to the centroid vertex. Com-
pared with hypergraph-based deep learning method HGNN,
our convolution module better fuses information from local
2 Related Work
and global features provided by our DHG module. In this section, we give a brief review on graph-based deep
We have applied our model to data with and without inher- learning and hypergraph learning.
ent graph structure. For data with inherent graph structure, we
conducted an experiment on a citation network benchmark, 2.1 Graph-based Deep Learning
the Cora dataset [Sen et al., 2008], for the node classifica- Semi-supervised learning on graphs has long been an active
tion task. In this experiment, we used DHGNN to jointly research field in deep learning. DeepWalk [Perozzi et al.,
learn embeddings from given graph structure and a hyper- 2014] and Planetoid [Yang et al., 2016] view sampled paths
graph structure from feature space. For data without inherent in graphs as random sequences and learn vector embedding
graph structure, an experiment was conducted on a social me- from these sequences.
dia dataset, the Microblog dataset [Ji et al., 2019], for the sen- After the great success of convolutional neural net-
timent prediction task. In this experiment, a multi-hypergrpah works [Krizhevsky et al., 2012] in image processing, re-
was constructed to model the complex relations among mul- searchers have been devoted to designing convolutional meth-
timodal data. ods for graph-based data. Existing graph neural network
Our contributions are summarized as follows: methods can be divided in two main categories: spectral
methods and spatial methods.
1. We propose a dynamic hypergraph construction method, Based on spectral graph theory, spectral graph convolu-
which adopts k-NN method to generate basic hyperedge tional methods use graph Laplacian eigenvectors as graph
and extends adjacent hyperedge set by clustering algo- Fourier basis. After transforming features to spectral do-
rithm, i.e., k-means clustering. By dynamic hypergraph main, a spectral convolution operation is conducted on
construction method, local and global relations will be spectral features. To overcome the expensive computation
extracted. cost in Laplacian factorization, ChebyshevNet introduces
2. We conducted experiments on network-based classifi- Chebyshev polynomials to approximate Laplacian eigenvec-
cation and social media sentiment prediction. On the tors [Defferrard et al., 2016]. GCN further simplifies the pro-
network-based task, our method outperforms state-of- cess and uses one-order polynomial on each layer [Kipf and
the-art methods and shows higher robustness to different Welling, 2017].

2636
 Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19)

Different from spectral methods, spatial graph convolution Algorithm 1 Hypergraph Construction
methods leverage spatial sampler and aggregator to generate Input: Input embedding X; hyperedge size k; adjacent hy-
neighborhood feature embedding. MoNet defines a generic peredge set size S
spatial convolution framework for deep learning on non- Output: Hyperedge set G
Euclidean domains [Monti et al., 2017]. GraphSAGE defines Function: k-Means clustering kM eans; k-nearest neighbor-
sampler and aggregator in graph neural network and tries hood selection knn; distance function dis; S − 1 smallest
LSTM as neighborhood aggregator [Hamilton et al., 2017]. distance index selection topK
GAT introduces self-attention mechanism and computes the 1: C = kM eans(X)
attention coefficients between node pairs [Veličković et al., 2: for u in range(len(X)) do
2018]. In the field of computer vision, DGCNN, a 3D point 3: eb = knn(X[u], X, k)
cloud learning method, also leverages the concept of spatial 4: G[u].insert(eb )
graph convolution in its model [Wang et al., 2018]. 5: D = dis(C.center, u)
6: D = sort(D)
2.2 Hypergraph Learning 7: ind = topK(D, S − 1)
Hypergraph learning is first introduced by [Zhou et al., 2007] 8: for i in ind do
as a label propagation method for semi-supervised learning. 9: G[u].insert(C[i])
This method aims to minimize the differences in labels of 10: end for
vertices that share the same hyperedge. [Huang et al., 2009] 11: end for
discusses the construction methods of hypergraph, including
k-NN method and search radius method. More recent works We use symbol Con(e) to denote the vertex set a hyper-
concentrate on the learning of hyperedge weight, intending to edge e contains and use symbol Adj(u) to denote the hyper-
assign larger weight to hyperedges or sub-hypergraphs with edge set composed of all hyperedge containing the vertex u,
higher importance [Gao et al., 2012]. Besides learning label which is formulated as:
propagation on hypergraph, dynamic hypergraph structure
learning proposes learning of hypergraph structure by a dual
optimization process [Zhang et al., 2018]. Like graph neu- Con(e) = {u1 , u2 , ..., uke } (1)
ral network, hypergraph neural network (HGNN) has been Adj(u) = {e1 , e2 , ..., eku } (2)
proposed as the first deep learning method on hypergraph where ke and ku is the number of vertices in hyperedge e and
structure, employing hypergraph Laplacian to represent hy- the number of hyperedges containing vertex u. Vertex u is
pergraph from spectral perspective [Feng et al., 2018]. defined as the centroid vertex of hyperedge set Adj(u).
Hypergraph has many aspects of applications. In computer We combine k-NN methods and k-means clustering meth-
vision, hypergraph is used to describe relations among visual ods for dynamic hypergraph construction to exploit local and
features for tasks like visual classification [Wang et al., 2015], global structure. On one hand, we have computed the k − 1
image retrieval [Huang et al., 2010] and video object seg- nearest neighbors for each vertex u. These neighborhood ver-
mentation [Huang et al., 2009]. There are also works using tices, along with the vertex u, form a hyperedge in Adj(u).
hypergraph structure for label propagation on 3D model clas- On the other hand, we have conducted k-means algorithm on
sification [Zhang et al., 2018]. In social media, MHG [Chen the whole feature map of each layer according to Euclidean
et al., 2015] and Bi-MHG [Ji et al., 2019] are proposed to distance. For each vertex, the nearest S −1 clusters will be as-
deal with multimodal data. signed as the adjacent hyperedges of this vertex. The detailed
procedure is described in Algorithm 1.
3 Dynamic Hypergraph Neural Networks We perform such procedure on the feature embedding of
each layer. Especially, we initialize hypergraph structure with
In this section, we introduce the dynamic hypergraph neural the input feature embedding. Therefore, the hyperedge set is
networks (DHGNN) proposed in detail. As is illustrated in dynamically adjusted as the feature embedding evolves with
Figure 2, a DHGNN layer consists of two major part: dy- network going deeper. In this way, we are able to obtain bet-
namic hypergraph construction (DHG) and hypergraph con- ter hypergraph struture for high-order data relation modeling
volution (HGC). We will first introduce these two parts in the with deep neural network.
following subsections and then discuss the implementation of
dynamic hypergraph neural networks in the last subsection. 3.2 Hypergraph Convolution
Hypergraph convolution is composed of two submodules:
3.1 Dynamic Hypergraph Construction vertex convolution submodule and hyperedge convolution
submodule. Vertex convolution aggregates vertex features to
Given feature embedding X = [x1 ; x2 ; ...; xn ] where xi (i =
hyperedge and then hyperedge convolution aggregates adja-
1, 2, ..., n) denotes the feature of the i-th sample, we construct
cent hyperedge features to centroid vertex by hyperedge con-
hypergraph G. In hypergraph, vertex u denotes a sample and
volution.
hyperedge e denotes a sample collection containing a flexible
number of samples. Therefore, a hypergraph can be formu- Vertex Convolution
lated as G = {V, E}, where V denotes the vertex set and E Vertex convolution aggregates vertex features to the hyper-
denotes the hyperedge set. edge containing these vertices. A simple solution is pooling

2637
 Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19)

Transform Algorithm 2 Hypergraph Convolution

Conv
matrix
Input: Input sample xu ; hypergraph structure G
Output: Output sample yu
Vertex Features Hyperedge
Feature 1: xlist = Φ
Conv 2: for e in Adj(u) do
Mul
3: Xv = V ertexSample(X, G)
4: xe = V ertexConv(Xv )
5: xlist.insert(xe )
6: end for
7: Xe = stack(xlist)
Figure 3: Vertex convolution module. For k vertices, a k × k trans-
form matrix is computed by convolution. We multiply transform 8: xu = EdgeConv(Xe )
matrix and input vertex feature matrix to get permuted and weighted 9: yu = σ(xu W + b)
vertex feature matrix. Then a 1-dimension convolution is conducted
for the 1-dimension hyperedge feature.
procedure can be formulated as follows:
MLP w

w = sof tmax(xe W + b) (4a)

Centroid
Hyperedge |Adj(u)|
Dot Vertex
Features X
Feature
xu = wi xie (4b)
i=0
Sum
...
...

|Adj(u)| denotes the size of adjacent hyperedge set, xe de-

notes adjacent hyperedge features and xu denotes centroid
vertex feature. W and b are learnable parameters.

3.3 Dynamic Hypergraph Neural Networks

Figure 4: Hyperedge convolution module. We use self-attention By combining vertex convolution and hyperedge convolu-
mechanism to aggregate hyperedge features. Self-attention weight tion, we describe a hypergraph convolutional layer as Al-
w is computed through multi-layer perception (MLP) from hyper-
gorithm 2. For each hyperedge e in Adj(u), we first sam-
edge features. Then weighted average of hyperedge features is cal-
culated as centroid vertex feature. ple k vertices in e and obtain Xu ∈ Rk×d , where d is in-
put dimension of feature. Vertex Convolution V ertexConv
transforms vertex features Xu to hyperedge feature xe ∈ Rd .
method like max pooling and average pooling. State-of-the- Aftering performing |Adj(u)| vertex convolution, we stack
art methods employ a fixed, pre-computed transform matrix |Adj(u)| hyperedge features to get adjacent feature matrix
generated from graph/hypergraph structure for vertex aggre- Xe ∈ R|Adj(u)|×d . Then Hyperedge Convolution EdgeConv
gation. However, such methods are unable to model discrim- aggregates adjacent hyperedge features to feature xu of ver-
inative information among vertices features well. To tackle tex u. At last, xu is updated to yu by a non-linear activation
this issue, we learn the transform matrix T from the vertex function σ following a linear layer.
features for feature permutation and weighting, as is shown A DHGNN model is composed of a stack of several layers
in Figure 3. The transform matrix enables both inter-vertex of dynamic hypergraph construction construction module and
and inter-channel information flow. In implementation, we hypergraph convolution module. As is shown in Figure 2,
use multi-layer perception (MLP) to generate transform ma- a hypergrpah convolutional layer updates vertex features for
trix T and use 1-d convolution to compact the transformed new feature embedding, based on which a new hypergraph
features, as is described by equation 3. structure will be constructed.

4 Applications and Experiments

T = M LP (Xu ) (3a)
In this section, we applied our dynamic hypergraph neural
xe = conv(T · M LP (Xu )) (3b)
network to two types of data: citation network with inherent
graph structure and social media multimodal data without in-
herent data structure. Especially, for data with inherent graph
Hyperedge Convolution structure, we sample k vertices in 1-order neighborhood of u
Hyperedge convolution aggregates hyperedge features to cen- and these k vertices also form a hyperedge in Adj(u).
troid vertex feature, as is illustrated in Figure 4. Atten-
sion mechanism is employed in hyperedge convolution, us- 4.1 Experiments on Citation Network
ing multi-layer perception (MLP) to generate weight score of Citation network is a typical graph-structure dataset. Since
each hyperedge. The output centroid vertex feature is com- graph is a special case of hypergraph, our model applies to
puted as a weighted sum of input hyperedge features. The graph-structure data as well. Furthermore, by constructing

2638
 Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19)

dynamic hypergraph, we are able to deploy information from lr #train GCN HGNN GAT DHGNN
both citation relation and feature embedding relation in a uni-
form manner. To evaluate the performance of our method, se- std 140 81.5% 81.6% 83.0% 82.5%
ries of experiments were conducted on the public benchmark 2% 54 69.6% 75.4% 74.8% 76.9%
citation network dataset, Cora dataset. 5.2% 140 77.8% 79.7% 79.4% 80.2%
10% 270 79.9% 80.0% 81.5% 81.6%
Cora dataset. Cora dataset is a benchmark dataset of cita- 20% 540 81.4% 80.1% 83.5% 83.6%
tion network. In Cora dataset, there are 2,708 vertices denot- 30% 812 81.9% 82.0% 84.5% 85.0%
ing academic papers and 5,429 edges denoting citation rela- 44% 1200 82.0% 81.9% 85.2% 85.6%
tions between pairs of papers. Each vertex has a bag-of-word
feature vector and a category label indicating the subject that Table 1: Performance comparisons on Cora with different splits. ”lr”
the paper belongs to. There are 7 categories in total. stands for label rate, ”#train” stands for number of training samples
Experimental setup. We have conducted experiments on and ”std” stands for standard split. Standard split experiment and
different splits of the Cora dataset including standard split 5.2% split experiment share the same number of training samples.
described in [Yang et al., 2016]. Because the standard split Different from the standard split setting, samples in 5.2% split is
randomly selected. 44% is the largest possible size of training set
uses fixed training samples with 5.2% of dataset, it is possible
with standard validation and test set.
for a method to be influenced by the fixed data distribution.
Therefore, for further comparison, we randomly sampled dif-
ferent proportion of data as training set to demonstrate the
effectiveness of our method. The proportion for training set
is selected as 2%, 5.2%, 10%, 20%, 30% and 44%, respec-
tively. We compared our method with recent representative
methods like GCN [Kipf and Welling, 2017], HGNN [Feng
et al., 2018] and GAT [Veličković et al., 2018]. 10 times av-
erage accuracy was reported in Table 1 for comparison.
We used 2 layer dynamic hypergraph neural network with
a GCN-style input layer for feature dimension reduction. We
used 400 cluster centers in k-means clustering method and Figure 5: Ablation experiment on dynamic hypergraph(DHG) mod-
chose 64 as the receptive field size. We added two dropout ule and hypergraph convolution (HGC) module. For model without
layers with the dropout rate of 0.5 before two convolutional dynamic hypergraph, we used inherent graph structure on Cora for
layers. convolution operation. For model without hypergraph convolution,
we used average pooling for substitution of vertex convolution and
Semi-supervised node classification. We compared
hyperedge convolution.
DHGNN with most recent graph/hypergraph-based neural
network methods on different dataset splits. Experimental
results are listed as Table 1, showing that our method outper- hypergraph-based learning method, i.e., HGNN and when
formed state-of-the-art by 1.5%, 0.5%, 0.1%, 0.1%, 0.5%, k = 128, our method outperforms HGNN by 0.9%. This
0.4% when 2%, 5.2%, 10%, 20%, 30%, 44% randomly sam- implies that our method is able to aggregate neighborhood
pled data was used as training set, respectively. Moreover, features better.
we observed that hypergraph structure was relatively more
competitive when training set was smaller. The reason is that 4.2 Microblog Sentiment Prediction
graph convolution only uses 1-order adjacent relation while Apart from experiments on citation network, we also eval-
hypergraph convolution utilizes high-order relation, which uated our model on a more complicated task, social media
is helpful to the label propagation process on a sparsely sentiment prediction. Multi-modality is an important feature
labelled graph. of social media. We used hypergraph to model the high-
Ablation experiments. To evaluate the effectiveness of order relations among different modalities. Specifically, ver-
proposed dynamic hypergraph construction (DHG) module tices were used to denote a tweet. Hyperedge sets were con-
and hypergraph convolution (HGC) module, we conducted structed according to each modality feature and hyperedges
two ablation experiments on the Cora dataset, where each from multi-modalities jointly represent the correlation be-
module mentioned above was removed from the complete tween vertices. In our experiment, we used the Microblog
model. We compared the ablated models against the com- dataset to evaluate our hypergraph model.
plete model and investigated the influence of hyperparameter Microblog dataset. The Microblog dataset contains 5,550
k, which denotes the number of sampled vertices in a hyper- tweets crawled from Sina Microblog platform 1 during Feb.
edge. Results are shown in Figure 5. From the results, we 2014 to Apr. 2014. Each tweet has three modalities: text,
observe that DHG module and HGC module always improve image and emoticon. We generated 2547-dimension bag-of-
the performance of baseline with different k. As k increases, words textual feature using Chinese auto-segmentation sys-
the gain from both modules increases, indicating the effec- tem ICTCLAS [Zhang et al., 2003]. To generate visual
tiveness of our method. It is noted that even when k = 4 feature, we used SentiBank [Borth et al., 2013], a kind
(is much smaller than the max degree in the Cora dataset,
1
169), our method still obtains similar performance with other https://www.weibo.com

2639
 Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19)

of ANP detector library pre-trained on Twitter images to Method Acc Train Time
transform Microblog images to 1553-dimension feature vec-
tor. For emoticons, we built a emoticons dictionary with 49 CBM-NB [Wang et al., 2014] 71.6% -
frequently-used emoticons and computed bag-of-emoticons CBM-LR [Wang et al., 2014] 79.9% -
features. Each tweet has a label indicating its emotional po- CBM-SVM [Wang et al., 2014] 81.6% -
larity (postive or negative). The task is to predict tweet emo- HGNN [Feng et al., 2018] 86.8% 2m11s
tional polarity by multimodal features. There are 4196 posi- MHG noW [Chen et al., 2015] 87.3% -
tive tweets and 1354 negative tweets in the dataset. MHG [Chen et al., 2015] 88.6% -
MMHG [Chen et al., 2015] 88.7% -
Experimental setup. We followed the experimental setup Bi-MHG [Ji et al., 2019] 90.0% 58.5h
in [Ji et al.], where 4,650, 400, 500 tweets were randomly
selected as training, validation and test set, respectively. 10- DHGNN (our method) 91.8% 1m32s
times average accuracy was reported for method evaluation.
We used 2 layer dynamic hypergraph neural network with a Table 2: Performance comparisons on the Microblog dataset
multi-input fully-connected layer for feature dimension re-
duction. Dimensions of each modality feature were reduced
convolution respectively. Despite of this, we also note that in
to 32 before hypergraph convolution. We constructed three
the standard split of Cora dataset, GAT performs better than
hyperedge sets for three modality respectively and merged
DHGNN. The main reason is that in standard split, training
these sets as one multimoal hyperedge set. For each modal-
set contains fixed samples, thus suffering from larger ran-
ity, we use 400 cluster centers in k-means clustering method
domness and bias. On the other settings, we have randomly
and the number of vertices contained is 8 in each clusters. We
sampled training set for 10 times and reported the average ac-
select 2 nearest clusters from k-means clusters and one k-NN
curacy for comparison to suppress such randomness and bias.
cluster as the adjacent hyperedge set of each vertex. We use
identical activation and dropout setting with Section 4.1. We Discussion on time complexity. Traditional hypergraph
compare our model with recent approaches for multimodal learning models like Bi-MHG involve iterative optimization
sentiment prediction, such as Multi-kernel SVM [Zhang et and matrix inversion, thus suffering from larger time cost
al., 2011], Cross-media Bag-of-words Model [Wang et al., than neural network model. When comparing HGNN and
2014], Bi-layer Multimodal Hypergraph Learning [Ji et al., DHGNN, we find that parameter number of both models
2019], etc. Experiments were conducted on a Nvidia GeForce 0.133M, indicating that it takes roughly the same time to
GTX 1080 Ti GPU with 11G memory and 10.6 T-flops com- train a HGNN/DHGNN epoch. However, it takes 30 epochs
puting capacity. on average for DHGNN to converge on Microblog sentiment
Microblog sentiment prediction. In this experiment, we dataset while it takes 200 epochs on average for HGNN to
ran DHGNN to fuse features from mutliple modalities for converge. Therefore, DHGNN runs faster on Microblog sen-
sentiment label prediction. Experimental results are shown timent dataset.
in Table 2, which can be summarized as:
1. In terms of prediction accuracy, DHGNN achieved 5 Conclusions
higher performance with 1.8% accuracy gains in multi-
modal sentiment prediction task compared with current In this paper, we propose a dynamic hypergraph neural net-
state-of-the-art method. works framework to update hypergraph structure on each
layer. The method consists of two important modules: dy-
2. In terms of time expense, DHGNN remarkably short- namic hypergraph construction method and hypergraph con-
ened training time compared with current state-of-the- volution, where hypergraph convolution includes vertex con-
art methods (2300 times as fast as Bi-MHG and 1.4 volution and hyperedge convolution for hypergraph neighbor-
times as fast as HGNN). hood feature aggregation. We apply our model to citation net-
Experimental results indicate that our method outper- work data and multimodal social media data for evaluation.
formed the state-of-the-art method in both prediction accu- Results demonstrate that our model achieves similar of bet-
racy and training speed. The experiments on Microblog ter performance compared with state-of-the-art methods and
dataset demonstrates the effectiveness of our method in mod- is more robust to different data distributions. We also inves-
eling the high-order relations among multimodal data. tigate the effectiveness of dynamic hypergraph construction
module and hypergraph convolution module independently
4.3 Discussion by ablation experiment. In our model, k-NN method and
Discussion on accuracy. Compared with statically initial- k-means clustering method are used in hypergraph dynamic
ized hypergraph structure, dynamic hypergraph structure can construction. Future work can concentrate on better and more
better represent data distribution in deeper layers. Compared interpretable hypergraph construction methods.
with pooling and multi-layer perception, hypergraph convolu-
tion uses fixed-size, weight-shared learnable convolution ker- Acknowledgments
nel for feature extraction, thus being better for information
aggregation. Ablation experiments demonstrate the effective- This work was supported by National Natural Science Funds
ness of the dynamic hypergraph construction and hypergraph of China (U1701262, U1801263, 61671267).

2640
 Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19)

References [Perozzi et al., 2014] Bryan Perozzi, Rami Al-Rfou, and

[Borth et al., 2013] Damian Borth, Rongrong Ji, Tao Chen, Steven Skiena. Deepwalk: Online learning of social repre-
sentations. In Proceedings of the 20th ACM SIGKDD in-
Thomas Breuel, and Shih-Fu Chang. Large-scale visual
ternational conference on Knowledge discovery and data
sentiment ontology and detectors using adjective noun
mining, pages 701–710. ACM, 2014.
pairs. In Proceedings of the 21st ACM international con-
ference on Multimedia, pages 223–232. ACM, 2013. [Sen et al., 2008] Prithviraj Sen, Galileo Namata, Mustafa
[Chen et al., 2015] Fuhai Chen, Yue Gao, Donglin Cao, and Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad.
Collective classification in network data. AI magazine,
Rongrong Ji. Multimodal hypergraph learning for mi-
29(3):93–93, 2008.
croblog sentiment prediction. In 2015 IEEE International
Conference on Multimedia and Expo (ICME), pages 1–6. [Veličković et al., 2018] Petar Veličković, Guillem Cucurull,
IEEE, 2015. Arantxa Casanova, Adriana Romero, Pietro Liò, and
Yoshua Bengio. Graph attention networks. International
[Defferrard et al., 2016] Michaël Defferrard, Xavier Bres-
Conference on Learning Representations, 2018.
son, and Pierre Vandergheynst. Convolutional neural net-
works on graphs with fast localized spectral filtering. In [Wang et al., 2014] Min Wang, Donglin Cao, Lingxiao Li,
Advances in neural information processing systems, pages Shaozi Li, and Rongrong Ji. Microblog sentiment analysis
3844–3852, 2016. based on cross-media bag-of-words model. In Proceedings
of international conference on internet multimedia com-
[Feng et al., 2018] Yifan Feng, Haoxuan You, Zizhao puting and service, page 76. ACM, 2014.
Zhang, Rongrong Ji, and Yue Gao. Hypergraph neural net-
works. AAAI 2019, 2018. [Wang et al., 2015] Meng Wang, Xueliang Liu, and Xindong
Wu. Visual classification by l1 -hypergraph modeling.
[Gao et al., 2012] Yue Gao, Meng Wang, Dacheng Tao, IEEE Transactions on Knowledge and Data Engineering,
Rongrong Ji, and Qionghai Dai. 3-d object retrieval and 27(9):2564–2574, 2015.
recognition with hypergraph analysis. IEEE Transactions
[Wang et al., 2018] Yue Wang, Yongbin Sun, Ziwei Liu,
on Image Processing, 21(9):4290–4303, 2012.
Sanjay E. Sarma, Michael M. Bronstein, and Justin M.
[Hamilton et al., 2017] Will Hamilton, Zhitao Ying, and Jure Solomon. Dynamic graph cnn for learning on point clouds.
Leskovec. Inductive representation learning on large CoRR, abs/1801.07829, 2018.
graphs. In Advances in Neural Information Processing
[Yang et al., 2016] Zhilin Yang, William W Cohen, and Rus-
Systems, pages 1024–1034, 2017.
lan Salakhutdinov. Revisiting semi-supervised learning
[Huang et al., 2009] Yuchi Huang, Qingshan Liu, and Dim- with graph embeddings. ICML, 2016.
itris Metaxas. Video object segmentation by hypergraph [Zhang et al., 2003] Hua-Ping Zhang, Hong-Kui Yu, De-Yi
cut. In 2009 IEEE Conference on Computer Vision and
Xiong, and Qun Liu. Hhmm-based chinese lexical an-
Pattern Recognition, pages 1738–1745. IEEE, 2009.
alyzer ictclas. In Proceedings of the second SIGHAN
[Huang et al., 2010] Yuchi Huang, Qingshan Liu, Shaoting workshop on Chinese language processing-Volume 17,
Zhang, and Dimitris N Metaxas. Image retrieval via prob- pages 184–187. Association for Computational Linguis-
abilistic hypergraph ranking. In 2010 IEEE Computer So- tics, 2003.
ciety Conference on Computer Vision and Pattern Recog- [Zhang et al., 2011] Daoqiang Zhang, Yaping Wang, Lup-
nition, pages 3376–3383. IEEE, 2010. ing Zhou, Hong Yuan, Dinggang Shen, Alzheimer’s Dis-
[Ji et al., 2019] Rongrong Ji, Fuhai Chen, Liujuan Cao, and ease Neuroimaging Initiative, et al. Multimodal classifica-
Yue Gao. Cross-modality microblog sentiment prediction tion of alzheimer’s disease and mild cognitive impairment.
via bi-layer multimodal hypergraph learning. IEEE Trans- Neuroimage, 55(3):856–867, 2011.
actions on Multimedia, pages 1062–1075, 2019. [Zhang et al., 2018] Zizhao Zhang, Haojie Lin, Yue Gao,
[Kipf and Welling, 2017] Thomas N. Kipf and Max Welling. and KLISS BNRist. Dynamic hypergraph structure learn-
Semi-supervised classification with graph convolutional ing. In IJCAI, pages 3162–3169, 2018.
networks. In International Conference on Learning Rep- [Zhou et al., 2007] Dengyong Zhou, Jiayuan Huang, and
resentations (ICLR), 2017. Bernhard Schölkopf. Learning with hypergraphs: Cluster-
[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, ing, classification, and embedding. In Advances in neural
and Geoffrey E Hinton. Imagenet classification with deep information processing systems, pages 1601–1608, 2007.
convolutional neural networks. In Advances in neural in-
formation processing systems, pages 1097–1105, 2012.
[Monti et al., 2017] Federico Monti, Davide Boscaini,
Jonathan Masci, Emanuele Rodola, Jan Svoboda, and
Michael M Bronstein. Geometric deep learning on graphs
and manifolds using mixture model cnns. In Proceedings
of the IEEE Conference on Computer Vision and Pattern
Recognition, pages 5115–5124, 2017.

2641