2023 International Conference on Algorithms, Computing and Data Processing (ACDP)

Sequential Feature Augmentation for Robust Text-to-SQL

Hao Shen Ran Shen

Marketing Department Marketing Department
State Grid Zhejiang Electric Power Co., Ltd State Grid Zhejiang Electric Power Co., Ltd
Hangzhou, China Hangzhou, China
2023 International Conference on Algorithms, Computing and Data Processing (ACDP) | 979-8-3503-2668-0/23/$31.00 ©2023 IEEE | DOI: 10.1109/ACDP59959.2023.00042

shen [email protected] shen [email protected]

Gang Sun Yiling Li

Marketing Department Marketing Department
State Grid Zhejiang Electric Power Co., Ltd State Grid Zhejiang Electric Power Co., Ltd
Hangzhou, China Hangzhou, China
sun [email protected] li [email protected]

Yifan Wang Pengcheng Zhang*

Marketing Department Zhejiang University
State Grid Zhejiang Electric Power Co., Ltd Hangzhou, China
Hangzhou, China [email protected]
wang [email protected]

Abstract—The task of converting natural language queries I. I NTRODUCTION

into SQL queries, known as Text-to-SQL, plays a crucial role
in bridging the gap between human language and database The ability to communicate with database systems using
systems. However, Text-to-SQL systems face numerous chal-
natural language queries has the potential to revolutionize
lenges due to the inherent complexity of natural language
understanding and the structural differences between text and the way we interact with data. Especially in enterprises
SQL. On the one hand, the accuracy of SQL generation of like the State Grid Corporation of China with numerous
previous Text-to-SQL models needs to be further improved. On users, the Text-to-SQL task becomes even more important.
the other hand, the robustness of previous Text-to-SQL models However, converting natural language queries into structured
is not enough, so the usability is not strong enough. In this
database queries, also known as Text-to-SQL, remains a
paper, we propose a novel approach called Sequential Feature
Augmentation Method (SFAM) to enhance the robustness of challenging task due to the complexity of natural language
Text-to-SQL models. In SFAM, we generate a corresponding understanding and the structural differences between text
augmented feature sequence for each preliminary feature and SQL.
sequence in the text encoding stage. To strengthen the ability of Recent advances in deep learning have shown promising
Text-to-SQL model to distinguish the original sequence features
and the augmented sequence features, we designed a method results in Text-to-SQL[1], [2], [3], [4], [5], [6], [7], [8],
based on sequential consistency learning. In addition, we design but the performance of existing models is still limited
a pooling method based on attention mechanism and mutual by their ability to handle variations in natural language
information maximization to reduce the loss of information queries and generate accurate SQL queries. Especially when
during pooling. Constrained by these methods during training, these models face data different from the training set, their
our model not only greatly increases robustness but also
achieves better accuracy of SQL generation. We construct a performance usually deteriorates, which means that the
dataset based on the marketing business data of the State Grid robustness of these models needs to be further enhanced.
Corporation of China. Experiments on this dataset demonstrate Shen et al. [8] decomposed the Text-to-SQL model into four
our SFAM can achieve state-of-the-art effects on the Text- subtasks: table selection, column selection, SQL generation,
to-SQL task. Besides, to verify the robustness of our model, and value filling. Through model decomposition, this method
we randomly sample some data in the test set of the above
dataset, and generate synthetic queries based on ChatGPT. (SPSQL) [8] greatly improves the accuracy of generating
Experiments on this test set verify the robustness of our SFAM. SQL. However, when faced with different types of queries,
the performance of SPSQL decreases due to its insufficient
Keywords-Text-to-SQL; Sequential Consistency; Mutual In-
robustness. The key to improve the robustness of the Text-to-
formation Maximization; Robustness;
SQL model is to improve the robustness of SQL generation
and value filling models. Based on this thought, in this paper,
*Corresponding Author we propose a novel approach called Sequential Feature

979-8-3503-2668-0/23/$31.00 ©2023 IEEE 217

DOI 10.1109/ACDP59959.2023.00042
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR. Downloaded on January 10,2025 at 13:00:11 UTC from IEEE Xplore. Restrictions apply.
 Augmentation Method (SFAM) to enhance the robustness II. R ELATED W ORK
of the Text-to-SQL model. This section briefly reviews some literature related to
The SQL generation and value filling models are essen- our work, namely, Text-to-SQL, Consistency Learning, and
tially two Encoder-Decoder model structures for Seq-to-Seq. Mutual Information Maximization.
The Seq-to-Seq model can only perform well if the text
queries are adequately encoded, so the encoding phase is A. Text-to-SQL
the key to improve the robustness of Seq-to-Seq model. Text-to-SQL is a very practical technology and has at-
In SFAM, we designed a sequential feature augmentation tracted the interest of many researchers[9], [10], [11], [12],
method in the text query encoding stage. That is, we [13], [14], [1], [2], [3], [4], [5], [6], [7], [8]. Especially with
adjust the original Encoder-Decoder model to the structure the rise of deep learning, the performance of Text-to-SQL
of Encoder-Augmentation-Decoder. We generate a corre- model has been greatly improved[1], [2], [3], [4], [5], [6],
sponding augmented feature sequence for each preliminary [7], [8]. These deep learning based models typically consist
encoded text sequence feature, and then both the original of two parts, an encoder for encoding text queries and a de-
feature sequence and the augmented feature sequence will coder for generating SQL. Now there is also a way to decom-
go through the subsequent encoding phase. The original pose Text-to-SQL tasks into multiple subtasks, which further
feature sequence after encoding and the augmented feature improves the accuracy of generating SQL by decomposing
sequence after encoding will be constrained by the sequential the Text-to-SQL model. Shen et al. decomposed the Text-
consistency learning based method, thus providing a more to-SQL model into four subtasks: table selection, column
robust encoding feature for the decoder. Besides, we design selection, SQL generation, and value filling. The column
a pooling method based on attention mechanism and mutual selection and SQL generation models are also essentially
information maximization to reduce the loss of information Encoder-Decoder structures. Through model decomposition,
during pooling. SFAM not only improves the robustness this method (SPSQL) greatly improves the accuracy of
of the Text-to-SQL model effectively, but also improves generating SQL. However, when faced with different types
the accuracy of SQL generation, because of sequential fea- of queries, the performance of existing models decreases due
ture augmentation, sequential consistency learning and our to its insufficient robustness. The key point of this paper is to
pooling method based on attention mechanism and mutual improve the robustness of the Encoder-Decoder model. We
information maximization. have specially designed a sequential feature augmentation
To verify the effectiveness of the SFAM method, we module between the encoder and the decoder. Combining the
construct a dataset based on the marketing business data constraints of the methods based on sequential consistency
of the State Grid Corporation of China. On this dataset, our learning and mutual information maximization, our method
SFAM achieves better results than previous state-of-the-art not only improves the robustness of the model, but also
methods. In addition, to verify the robustness of the SFAM further improves the accuracy of SQL generation.
method, we randomly extracted 500 pieces of data in the
test set of this dataset, generated and replaced text queries B. Consistency Learning
using ChatGPT, and then provided these synthetic queries Consistency learning has been used in many tasks. Zhou
to the model for testing. The experimental results show that et al.[15] designed consistency between instances of the
our SFAM method effectively improves the robustness of same category to learn intensive cross-instance correspon-
the model. dence. Zhu et al.[16] used consistency learning to design
In summary, the main contributions of our paper are as a method called CycleGAN for image-to-image translation.
follows: Kong et al.[17] use consistency learning at different stages
• We proposed a novel approach called Sequential Fea- of the video encoding process, which greatly improves the
ture Augmentation Method (SFAM) for Text-to-SQL performance of video representation learning. Jin et al.[18]
task, which enhances the robustness of Text-to-SQL learned better video-text correspondence using the cross-
model through the sequential feature augmentation modal graph consistency method and achieved gratifying
method. results in video-text retrieval tasks. Inspired by the above
• In the process of model training, we designed a method work[15], [16], [17], [18], we designed a sequential con-
based on sequential consistency learning and a pooling sistency learning based method in the Text-to-SQL task.
method based on mutual information maximization to The constraint of this method can effectively enhance the
constrain the relationship between the original feature robustness of the model.
sequence and the augmented feature sequence.
• The experimental results show that our SFAM method C. Mutual Information Maximization
not only effectively improves the robustness of Text-to- Mutual information maximization is the content of Infor-
SQL model, but also further improves the accuracy of mation Theory[19], and its related applications have received
SQL generation. more and more attention. Belghazi et al.[20] proposed a

218

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR. Downloaded on January 10,2025 at 13:00:11 UTC from IEEE Xplore. Restrictions apply.
 select number of high-voltage users
from workload of meter readers
Selection
How many high-voltage users

where meter reader='Sam'

Table
did Sam record

Augmentation

Augmentation
Sequential

Sequential
Decoder

Decoder
Encoder

Encoder
Feature

Feature
Selection
Column

SQL Generation Value Filling

prelimanary subsequent
encoding encoding decoding
pooling
augmentation

D+
feature

D-
D-
D+

close subsequent
pooling
encoding
distant
Sequential Feature Augmentation

Figure 1. Overview framework of our SFAM.

neural estimator called MINE, which is a novel method for III. M ETHOD
estimating mutual information designed based on gradient
descent in the corresponding neural network. There are In this paper, we propose a novel approach called Sequen-
also application cases of mutual information maximization tial Feature Augmentation Method (SFAM) for Text-to-SQL.
theory in graph neural networks and computer vision. DGI In SFAM, we designed a sequential feature augmentation
was proposed by Veliković et al.[21], which maximizes the module between the encoder and the decoder, which can
mutual information between the node characteristics and effectively improve the robustness of the Text-to-SQL model
the whole graph representation for better representations through constraints based on sequential consistency learning
of graph nodes. Ren et al.[22] maximizes the use of mu- and mutual information maximization methods. The overall
tual information for heterogeneous graph. Zhang et al.[23] architecture of SFAM is shown in the Figure1.
designed a pooling method based on maximizing mutual
information in cross-modal retrieval, which can effectively A. Step-by-step Text-to-SQL Model
maintain the information of the original map nodes in the Our work is based on the work of Shen et al. (SPSQL) [8],
pooled features. Hjelm et al.[24] designed a method based which is a step-by-step Text-to-SQL model. In this model,
on mutual information maximization to achieve better high the Text-to-SQL task is divided into four subtasks:
dimensional representations in the model. In SFAM, we
• Table Selection: The purpose of table selection is to
introduce a pooling method based on mutual information
select the correct table, and a Bert model is used in the
maximization in the sequence feature augmentation module,
table selectio subtask.
which effectively enhances the robustness of Text-to-SQL
• Column Selection: The purpose of column selection is
model.
to select correct columns, and a Bert model is used in
the column selection subtask.

219

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR. Downloaded on January 10,2025 at 13:00:11 UTC from IEEE Xplore. Restrictions apply.
 Original Feature where
0 1 2 3 4 5 exp(sim(so , si ))
Sequence βi = N (3)
si ∈Sall exp(sim(so , si ))
T
u v
where sim(u, v) = ||u||||v|| means the cosine similarity
Augmented
Feature Sequence
3 4 5 0 1 2 between the two vectors u and v. βi is the the similarity
distribution which signifies the proximity between so and
Figure 2. The feature augmentation method in our SFAM. each si ∈ Sall .
Then we utilize the feature-pairs (so , sj ) between so and
Sos to apply with contrastive loss, which is defined similar
• SQL Generation: The purpose of SQL generation is to InfoNCE [28] as follows:
to generate SQL without values, which is essentially exp(sim(so , sj )/τ )
an Encoder-Decoder model, and the SQL generation Loc = −log N (4)
si ∈Sall exp(sim(so , si )/τ )
subtask is made up of a T5 model.
• Value Filling: The purpose of value filling is to convert where τ is a temperature parameter. Minimizing Loc en-
SQL without values into SQL with values, which is courages the network to make the pooling original feature
essentially an Encoder-Decoder model, and the value so close to Sos , and distant to Sas .
filling subtask is made up of a T5 model. In the same way, loss Lac of augmented consistency can
In SFAM, we mainly explore how to enhance the robust- also be obtained. Then we obtain the sequential consistency
ness of the encoder-decoder model structure. Our work is learning loss Lsc as follows:
based on the work of Shen et al [8].
Lsc = Loc + Lac (5)
B. Feature Augmentation Method
D. Pooling based on Mutual Information Maximization
In the field of computer vision, data augmentation has
In the SFAM method, we design a pooling method based
shown a powerful effect[25], [26], [27]. Inspired by these
on mutual information maximization to reduce the loss of
works[25], [26], [27], we designed a sequential feature
information of the feature sequence after pooling. In the
augmentation method for Text-to-SQL. Specifically, for a
following, we will introduce our method by taking the
feature sequence, our method first segments it, and then
pooling method in the direction of the original feature
randomly shuffles these feature sequence fragments. Our
sequence as an example.
feature augmentation method is shown in Figure2.
Let the original feature sequence and the augmented
C. Sequential Consistency Module feature sequence after preliminary encoding in the T5 model
The augmented feature sequence has different information be denoted as Sop and Sap , respectively. The pooling original
from the original feature sequence because the order of feature so can be calculated with pooling method based on
the internal feature fragments is disrupted. If our model attention mechanism as follows:

can enhance the ability to distinguish between the original so = αi si (6)
feature sequence and the augmented feature sequence, the si ∈Sos
performance of the model will obviously be enhanced. Based
on this idea, we design a module based on sequential consis- exp(W2 σ(W1 si + b1 ) + b2 )
αi =  (7)
tent learning. This process is divided into two parts: original sj ∈Sos exp(W2 σ(W1 sj + b1 ) + b2 )
feature consistency and enhanced feature consistency, and
where σ is the activation function, W1 and W2 are two
the original feature consistency process is as follows:
learnable transformation weights matrices, and b1 and b2
Let the original feature sequence and the augmented
are two biases.
feature sequence after subsequent encoding in the T5 model
To reduce the loss of information of the feature sequence
be denoted as Sos and Sas , respectively. Assume that a feature
after pooling, we design strategies based on mutual in-
set is constructed as Sall = {Sos , Sas }. Then we calculate the
formation maximization. In order to maximize the mutual
pooling original feature so as follows:
information between so and Sos , same as Veliković et al.[21],
so = pool(Sos ) (1) we employ a discriminator D, and the discriminator is a
bilinear layer:
where pool is is the pooling operation along temporal
dimension. D(si , so ) = σ((ssi )T WD sp ) (8)
The soft nearest neighbor ŝ is defined as follows:
where WD is a learnable transformation weights matrix.
N
 Veliković et al.[21] theoretically proves that the bidi-
ŝ = βi si (2) rectional cross-entropy loss function can maximize mutual
si ∈Sall

220

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR. Downloaded on January 10,2025 at 13:00:11 UTC from IEEE Xplore. Restrictions apply.
 Table I Table II
OVERALL COMPARISON EXPERIMENT. ROBUSTNESS COMPARISON EXPERIMENT.

Model Logic Form Accuracy Model Logic Form Accuracy

IRNET [29] 36.2% IRNET [29] 32.3%
IGSQL [30] 69.1% IGSQL [30] 63.8%
RAT-SQL [31] 81.9% RAT-SQL [31] 77.8%
SPSQL [8] 94.7% SPSQL [8] 91.6%
SFAM (our method) 95.9% SFAM (our method) 95.1%

Table III
A BLATION EXPERIMENTS ON ROBUSTNESS COMPARISON .
information. To maximize the mutual information between
so and Sos and minimize the mutual information between so Model Logic Form Accuracy
and Sop , we use the bidirectional cross-entropy loss function
SPSQL [8] 91.6%
in the work of Veliković et al.[21] to achieve this process: average pooling 93.7%
1   w/o. MI 92.8%
LD = ( E[logD(si , so )]+ E[1−logD(sj , so )]) SFAM (our method) 95.1%
2T s p
si ∈So sj ∈So
(9)
where T is the number of features in Sos .
because of the role of the sequential feature augmentation
In the same way, loss L˜D in the direction of the original
module in our SFAM method to achieve such an effect.
feature sequence can be obtained. So the loss function of
To verify the robustness of our SFAM method, we design
the pooling part can be expressed as:
the experiments shown in Table II. Compared with the
LM I = LD + L˜D (10) results in Table I, it can be seen that when other models
[29], [30], [31], [8] encounter different types of text queries,
E. Overall Formulation the accuracy of their SQL generation will be significantly
Assume that the original calculation function of the model reduced, while our SFAM method can almost maintain the
is expressed as Lbase , the overall loss function L is ex- accuracy of SQL generation. The experimental results in
pressed as: Table II demonstrate the strong robustness of our SFAM
L = Lbase + λLsc + μLM I (11) method.

IV. E XPERIMENT C. Ablation Studies

To explore the role of each module in the SFAM method,
A. Implementation Details
we design experiments as shown in Table III. In Table III,
We conducted experiments in this paper by constructing we conducted experiments by replacing the pooling method
a dataset based on data from the marketing scenario of State based on the attention mechanism with average pooling and
Grid Corporation of China. The database in this dataset removing the constraint of mutual information maximiza-
contains 37 data tables, and this dataset contains a total of tion. Obviously, these operations will lead to a decrease
12,000 question-SQL pairs. The dataset was divided into in the accuracy of SQL generation. Table III illustrates the
10,000 training set samples and 2,000 test set samples. necessity of these modules in the SFAM method.
In particular, we randomly selected 500 samples from the To determine the appropriate values of hyperparameters λ,
test set, and replaced their corresponding text queries with μ, and τ , we conducted an experiment as shown in Figure
generated synonymous sentences based on ChatGPT, and 3. According to the results, we find λ = 1.0, μ = 0.1, and
used the replaced 500 question-SQL pairs as the test set to τ = 0.1 are the relatively appropriate values.
verify the robustness of the model.
Our experiment is carried out on the PyTorch framework. V. C ONCLUSION
In our experiments, Logic Form Accuracy is used to evaluate In this paper, we have proposed a novel approach, Se-
the accuracy of SQL generation. In SFAM, the hyperparam- quential Feature Augmentation Mehod (SFAM), to enhance
eters are set to λ = 1.0, μ = 0.1, and τ = 0.1. Other details the robustness of the Text-to-SQL model. By designing
of the network structure are the same as Shen et al[8]. a sequential feature augmentation method, our SFAM can
effectively improve the ability to distinguish between the
B. Comparison with the Existing Methods original feature sequence and the enhanced feature sequence,
As shown in Table I, compared with other state-of-the-art thereby improving the robustness of the SQL generation of
Text-to-SQL models[29], [30], [31], [8], our SFAM method the Text-to-SQL model. Experiments based on data from the
further improves the accuracy of SQL generation. This is State Grid Corporation of China show that our SFAM can

221

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR. Downloaded on January 10,2025 at 13:00:11 UTC from IEEE Xplore. Restrictions apply.
 濌濉澸 濌濉澸 濌濉澸

濌濈澸 濌濈澸
濌濈澸
濌濈澸 濌濈澸
濌濇澸
濌濇澸 濌濇澸

濌濆澸 濌濇澸 濌濇澸

濌濆澸 濌濆澸
濌濅澸
濌濆澸 濌濆澸
濌濄澸
濌濅澸 濌濅澸

濌濃澸 濌濅澸 濌濅澸

濃 濃濁濈 濄 濄濁濈 濅 濅濁濈 濃 濃濁濃濈 濃濁濄 濃濁濄濈 濃濁濅 濃濁濅濈 濃濁濃濈 濃濁濄 濃濁濄濈 濃濁濅 濃濁濅濈

(a) The influence of different λ (b) The influence of different μ (c) The influence of different τ

Figure 3. Ablation experiments on robustness comparison.

not only improve the robustness of the model but also further [10] F. Li and H. V. Jagadish, “Constructing an interactive natural
improve the accuracy of the model SQL generation. In the language interface for relational databases,” Proceedings of
future, we will consider applying our SFAM to practical the VLDB Endowment, vol. 8, no. 1, pp. 73–84, 2014.
scenarios. [11] L. Wan, Q. Wang, A. Papir, and I. L. Moreno, “General-
ized end-to-end loss for speaker verification,” in 2018 IEEE
ACKNOWLEDGMENT International Conference on Acoustics, Speech and Signal
This work is supported by Zhejiang Electric Power Co., Processing (ICASSP). IEEE, 2018, pp. 4879–4883.
Ltd. Science and Technology Project (No. 5211YF220006).
[12] A.-M. Popescu, A. Armanasu, O. Etzioni, D. Ko, and
R EFERENCES A. Yates, “Modern natural language interfaces to databases:
Composing statistical parsing with semantic tractability,” in
[1] L. Dong and M. Lapata, “Language to logical form with COLING 2004: Proceedings of the 20th International Con-
neural attention,” arXiv preprint arXiv:1601.01280, 2016. ference on Computational Linguistics, 2004, pp. 141–147.
[2] C. Wang, M. Brockschmidt, and R. Singh, “Pointing out sql [13] C. Unger, L. Bühmann, J. Lehmann, A.-C. Ngonga Ngomo,
queries from text,” 2018. D. Gerber, and P. Cimiano, “Template-based question answer-
ing over rdf data,” in Proceedings of the 21st international
[3] V. Zhong, C. Xiong, and R. Socher, “Seq2sql: Generating conference on World Wide Web, 2012, pp. 639–648.
structured queries from natural language using reinforcement
learning,” arXiv preprint arXiv:1709.00103, 2017. [14] C. Jinchao, H. Tao, C. Gang, W. Xiaofan, and C. Ke,
“Research on technology of generating multi-table sql query
[4] X. Xu, C. Liu, and D. Song, “Sqlnet: Generating structured statement by natural language,” Journal of Frontiers of Com-
queries from natural language without reinforcement learn- puter Science & Technology, vol. 14, no. 7, p. 1133, 2020.
ing,” arXiv preprint arXiv:1711.04436, 2017.
[15] T. Zhou, P. Krahenbuhl, M. Aubry, Q. Huang, and A. A.
[5] Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, Efros, “Learning dense correspondence via 3d-guided cycle
M. Lewis, L. Zettlemoyer, and V. Stoyanov, “Roberta: A consistency,” in Proceedings of the IEEE conference on
robustly optimized bert pretraining approach,” arXiv preprint computer vision and pattern recognition, 2016, pp. 117–126.
arXiv:1907.11692, 2019.
[16] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-
[6] Z. Yang, Z. Dai, Y. Yang, J. Carbonell, R. R. Salakhutdinov, to-image translation using cycle-consistent adversarial net-
and Q. V. Le, “Xlnet: Generalized autoregressive pretraining works,” in Proceedings of the IEEE international conference
for language understanding,” Advances in neural information on computer vision, 2017, pp. 2223–2232.
processing systems, vol. 32, 2019.
[17] Q. Kong, W. Wei, Z. Deng, T. Yoshinaga, and T. Murakami,
[7] Z. Zhang, X. Han, Z. Liu, X. Jiang, M. Sun, and Q. Liu, “Cycle-contrast for self-supervised video representation learn-
“Ernie: Enhanced language representation with informative ing,” Advances in Neural Information Processing Systems,
entities,” arXiv preprint arXiv:1905.07129, 2019. vol. 33, pp. 8089–8100, 2020.
[8] R. Shen, G. Sun, H. Shen, Y. Li, L. Jin, and H. Jiang, [18] W. Jin, Z. Zhao, P. Zhang, J. Zhu, X. He, and Y. Zhuang,
“Spsql: Step-by-step parsing based framework for text-to-sql “Hierarchical cross-modal graph consistency learning for
generation,” in CMVIT 2023 conference proceeding, 2023. video-text retrieval,” in Proceedings of the 44th International
ACM SIGIR Conference on Research and Development in
[9] X. Meng and S. Wang, “Nchiql: The chinese natural language Information Retrieval, 2021, pp. 1114–1124.
interface to databases,” in Database and Expert Systems
Applications: 12th International Conference, DEXA 2001 [19] C. E. Shannon, “A mathematical theory of communication,”
Munich, Germany, September 3–5, 2001 Proceedings 12. ACM SIGMOBILE mobile computing and communications
Springer, 2001, pp. 145–154. review, vol. 5, no. 1, pp. 3–55, 2001.

222

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR. Downloaded on January 10,2025 at 13:00:11 UTC from IEEE Xplore. Restrictions apply.
 [20] M. I. Belghazi, A. Baratin, S. Rajeshwar, S. Ozair, Y. Bengio,
A. Courville, and D. Hjelm, “Mutual information neural
estimation,” in International conference on machine learning.
PMLR, 2018, pp. 531–540.

[21] P. Velickovic, W. Fedus, W. L. Hamilton, P. Liò, Y. Bengio,

and R. D. Hjelm, “Deep graph infomax.” ICLR (Poster),
vol. 2, no. 3, p. 4, 2019.

[22] Y. Ren, B. Liu, C. Huang, P. Dai, L. Bo, and J. Zhang,

“Heterogeneous deep graph infomax,” arXiv preprint
arXiv:1911.08538, 2019.

[23] P. Zhang, Z. Zhao, N. Wang, J. Yu, and F. Wu, “Local-

global graph pooling via mutual information maximization
for video-paragraph retrieval,” IEEE Transactions on Circuits
and Systems for Video Technology, vol. 32, no. 10, pp. 7133–
7146, 2022.

[24] R. D. Hjelm, A. Fedorov, S. Lavoie-Marchildon, K. Grewal,

P. Bachman, A. Trischler, and Y. Bengio, “Learning deep
representations by mutual information estimation and max-
imization,” arXiv preprint arXiv:1808.06670, 2018.

[25] P. Bachman, R. D. Hjelm, and W. Buchwalter, “Learning

representations by maximizing mutual information across
views,” Advances in neural information processing systems,
vol. 32, 2019.

[26] T. Chen, S. Kornblith, K. Swersky, M. Norouzi, and

G. E. Hinton, “Big self-supervised models are strong semi-
supervised learners,” Advances in neural information process-
ing systems, vol. 33, pp. 22 243–22 255, 2020.

[27] X. Chen, H. Fan, R. Girshick, and K. He, “Improved base-

lines with momentum contrastive learning,” arXiv preprint
arXiv:2003.04297, 2020.

[28] A. Van den Oord, Y. Li, O. Vinyals et al., “Representation

learning with contrastive predictive coding,” arXiv preprint
arXiv:1807.03748, vol. 2, no. 3, p. 4, 2018.

[29] J. Guo, Z. Zhan, Y. Gao, Y. Xiao, J.-G. Lou, T. Liu, and

D. Zhang, “Towards complex text-to-sql in cross-domain
database with intermediate representation,” arXiv preprint
arXiv:1905.08205, 2019.

[30] Y. Cai and X. Wan, “Igsql: Database schema interaction

graph based neural model for context-dependent text-to-sql
generation,” arXiv preprint arXiv:2011.05744, 2020.

[31] B. Wang, R. Shin, X. Liu, O. Polozov, and M. Richardson,

“Rat-sql: Relation-aware schema encoding and linking for
text-to-sql parsers,” arXiv preprint arXiv:1911.04942, 2019.

223

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR. Downloaded on January 10,2025 at 13:00:11 UTC from IEEE Xplore. Restrictions apply.