Received 4 November 2023, accepted 15 November 2023, date of publication 21 November 2023,

date of current version 29 November 2023.

Digital Object Identifier 10.1109/ACCESS.2023.3335905

A Multitask Learning Framework for Predicting

Ship Fuel Oil Consumption
LOUKAS ILIAS , PANAGIOTIS KAPSALIS, SPIROS MOUZAKITIS , AND DIMITRIS ASKOUNIS
Decision Support Systems Laboratory, School of Electrical and Computer Engineering, National Technical University of Athens, 15780 Athens, Greece
Corresponding author: Loukas Ilias ([email protected])
This work was supported by the European Union through the Horizon 2020 Research and Innovation Program, in the Context of the VesselAI
Project under Agreement GA 957237.

ABSTRACT Predicting the ship fuel consumption constitutes a prerequisite for speed, trim, and voyage
optimization. In spite of the rise of deep learning and transformers in many domains, research works train
shallow machine learning (ML) algorithms for predicting ship fuel oil consumption (FOC). Although the
auxiliary machinery is in support of the main propulsion engines and the emissions from ships’ auxiliary
engines contribute to the environmental pollution, most existing research initiatives train ML algorithms
for predicting only the main engine FOC. Additionally, all the existing research initiatives use the mean
squared error (MSE) as the loss function. However, recent studies have shown that neural network models
tend to replicate the last observed value of the time series, thus limiting their applicability to real-world data.
To address these limitations, this is the first study proposing transformer-based approaches and a multitask
learning (MTL) framework. Firstly, the authors introduce Single-Task learning (STL) models consisting of
BiLSTMs and MultiHead Self-Attention for predicting the main and auxiliary engine FOC. Secondly, the
authors introduce the first MTL setting, which predicts the main and auxiliary engine FOC simultaneously
allowing one task to inform the other. A loss function is introduced, which includes a regularization term for
penalizing the replication of previously seen values. The authors evaluate the proposed approaches using data
from three fishing ships and compare these approaches with traditional ML algorithms. Extensive experiments
show that the introduced MTL models can improve the R2 score, mean bias error, root mean squared error,
and mean absolute error in comparison with shallow ML algorithms.

INDEX TERMS Maritime industry, main engine fuel oil consumption, auxiliary engine fuel oil consumption,
deep learning, MultiHead self-attention, multi-task learning, loss function.

LIST OF ABBREVIATIONS AND ACRONYMS 50% and 250% by 2050, if business goes as usual, undermining
In this section, the definitions of abbreviations and acronyms the objectives of the Paris Agreement. The study of [2] also
used in the paper are summarized in Table 1. Each abbreviation points out that ships consume large amounts of fuel oil and
and acronym is ordered from A to Z. consequently GHGs are released causing serious damage
to the environment, climate, and human health. Wave and
I. INTRODUCTION wind conditions affect significantly the fuel consumption.
The maritime vessels emit around 940 million tonnes of Specifically, the study in [3] analyzed changes in wind and
CO2 annually and are responsible for about 2.70% of global wave conditions over the last 27 years. After conducting
greenhouse gas (GHG) emissions [1]. Even worse, according numerical simulations for six voyages, the authors stated
to the 3rd International Maritime Organization (IMO) GHG that the difference between fuel consumption, travel time,
study, shipping emissions are projected to increase between average ship speed is significant for to and fro direction. For
instance, travelling from ‘‘Sydney to Valparaiso’’ requires
The associate editor coordinating the review of this manuscript and more time and fuel than ‘‘Valparaiso to Sydney’’ in both
approving it for publication was Seifedine Kadry . summer and winter seasons due to the direction of wave
2023 The Authors. This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.
132576 For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/ VOLUME 11, 2023
L. Ilias et al.: Multitask Learning Framework for Predicting Ship Fuel Oil Consumption

and wind. Slow steaming, i.e., sailing at reduced speed. TABLE 1. List of abbreviations and acronyms.
constitutes one of the measures proposed throughout the
years for mitigating CO2 emissions [4], [5]. Slow steaming
seems to be effective, since ship resistance is lower at the
reduced speed. However, according to the study [4], one should
analyze several locations along the sailing route and determine
local sea states for quantifying the benefits of slow steaming
more accurately. Additionally, sailing at reduced speed often
leads to a deterioration of the main and auxiliary engines
[6]. Apart from the careful examination of the operation of
the main engine under lower loads, one should also carefully
examine the operation of auxiliary engines, since the electrical
consumption of auxiliary engines has a significant role in
fuel consumption under slow steaming conditions [7]. Trim
optimization can also lead to savings in fuel consumption.
Grlj et al. [8] conducted a study for a containership and
stated that trim has an effect on wind and air resistance.
Specifically, findings suggested that trim by bow causes lower
values of air resistance, while trim by stern leads to higher
values in comparison to an even keel condition. To reduce
CO2 emissions in maritime transport, the IMO has introduced
the Energy Efficiency Design Index (EEDI), which is used
for new ships solely. Contrary to EEDI, the IMO has also
introduced the Energy Efficiency Existing Ship Index (EEXI)
[9]. Specifically,1 from 1 January 2023 it is mandatory for all
ships to calculate their attained EEXI to measure their energy
efficiency.
Although deep learning approaches and transformer-based
models are used in many tasks outperforming the traditional
machine learning algorithms, existing research initiatives
still train shallow machine learning regressors for predicting
fuel oil consumption. In addition, only few works have
proposed methods for estimating the auxiliary engines fuel These models comprise a BiLSTM layer and a MultiHead
oil consumption [10]. Although the auxiliary machinery Self-Attention layer for allowing the model to jointly attend
is in support of the main propulsion engines [11] and to information from different representation subspaces at
the emissions of ships’ auxiliary engines contribute to the different positions. Secondly, motivated by the fact that
environmental pollution, existing research initiatives train multitask learning (MTL) has been proved to be effective in
machine learning algorithms for predicting only the main many domains [16], [17], [18], including both related and
engine fuel consumption. Additionally, existing research unrelated tasks [19], the authors introduce the first MTL
initiatives use limited sets of features for training machine framework consisting of a primary task, i.e., prediction of
learning algorithms and predicting fuel oil consumption [12], main engine fuel oil consumption, and auxiliary task, i.e.,
[13], [14]. In addition, all the existing research initiatives prediction of auxiliary engine fuel oil consumption. MTL
minimize the mean squared error (MSE) loss for predicting allows tasks to be learned jointly, thus sharing knowledge
the fuel oil consumption. However, according to [15], one and features between the tasks. To address the phenomenon
limitation of this approach is the fact that the model just of ‘‘mimicking’’ in time series forecasting, the authors
replicates the last observed value of the time series. The add a regularization term in the loss function. the authors
authors in [15] define this problem as ‘‘mimicking’’ in time- exploit sensor data with a great number of features from
series forecasting. three fishing ships for conducting the experiments. The
To tackle the aforementioned limitations, this is the first authors train shallow machine learning algorithms, including
study employing BiLSTM and MultiHead Self-Attention BaggingRegressor, RandomForestRegressor, etc. and use them
layers in a multitask learning setting. Firstly, the authors as baselines. Findings suggest that the introduced approaches
introduce single-task learning (STL) models which predict offer valuable advantages over state-of-the-art ones.
the main and auxiliary engine fuel oil consumption separately. The main contributions of this study can be summarized as
follows:
1 https://www.imo.org/en/MediaCentre/HotTopics/Pages/EEXI-CII- • The authors utilize a multihead self-attention mechanism
FAQ.aspx for allowing the model to jointly attend to information

VOLUME 11, 2023 132577

 L. Ilias et al.: Multitask Learning Framework for Predicting Ship Fuel Oil Consumption

from different representation subspaces at different Reference [25] trained and compared multiple regression
positions. models for predicting CO2 emissions. The authors used a
• There is no prior work proposing a multitask learning set of 22 features, namely shaft generator power, speed over
framework for predicting main and auxiliary engine fuel ground, arrival draught, departure/arrival trim, etc. Next,
oil consumption simultaneously. the authors exploited variable selection approaches (forward
• This is the first study addressing the problem of stepwise regression), penalized regression models, latent
‘‘mimicking’’ in maritime series data by adding a variable methods, and tree-based ensemble methods.
regularization term in the loss function. An artificial neural network was also proposed by
• The authors use data from three vessels and exploit a [13]. After using data denoising methods, data clustering
large number of features in comparison with existing approaches, and data compression & expansion methods,
research initiatives. the authors employed an ANN for predicting the ship’s fuel
• The authors compare their proposed approaches with consumption. The authors used seven features, including the
shallow machine learning algorithms and show that the average draft, trim, main engine power, shaft speed, speed
introduced architectures outperform the traditional ones. through water, Speed over Ground (SOG), and relative wind
speed in knots. Results showed that ANN is a more accurate
II. RELATED WORK and efficient model to predict the fuel consumption of the
A. FUEL OIL CONSUMPTION PREDICTION TASK main engine than polynomial regression and support vector
Reference [20] used two types of datasets, namely noon- machine.
reports and Automated Data Logging & Monitoring systems In [26], the authors used data from noon reports, engine
for predicting main engine fuel oil consumption (FOC). logbook, and sensors to predict fuel oil consumption. They
The authors exploited a limited set of feature set consisting used the following features as inputs to machine learning
of 12 features, including vessel speed, draft aft, engine algorithms: Bearing temperatures, Fuel mass flow, Air coolers
speed, etc. Finally, the authors trained multiple regression cooling water temperature, Shaft power, and more. Finally,
algorithms, including Support Vector Machines (SVMs), the authors trained Multiple Linear Regression, Kernel Ridge
Random Forest Regressors (RFRs), Extra Trees Regressors Regression, Ridge and Lasso Regression, Support Vector
(ETRs), AdaBoost Regressors, Artificial Neural Networks Regression (SVR), Tree-Based Regression as Random Forest
(ANNs), Linear Regressors (LR), Ridge & Lasso Regressors, Regression and Decision Tree Regression and Boosting
and k-Nearest Neighbours Regressors. The authors stated that Algorithms including AdaBoost Algorithm and Gradient
the proposed models can accurately predict the FOC of vessels Boost Algorithm.
sailing under different conditions. In [27], the authors introduced a hybrid machine learning
The study in [21] used noon-reports and exploited an ANN model consisting of a Long Short-Term Memory (LSTM)
consisting of one hidden layer with 12 units for predicting fuel Recurrent Neural Network (RNN) and an Elman Neural
consumption. The authors used seven input variables, namely Network (ENN). The authors used as features the number
ship speed, revolutions per minute (RPM), mean draft, trim, of passengers, the average speed of the vessel, the wind force,
cargo quantity on board, wind and sea effects. etc. The authors compared the proposed architecture with:
The authors in [22] introduced a publicly available set of Radial Basis Function Neural Network (RBFNN), General
high-quality sensory data collected from a ferry over a period Regression Neural Network (GRNN), Elman Neural Network
of two months. The authors introduced a non-linear ANN (ENN), SVR, Group Method of Data Handling (GMDH)-
model to model ship fuel consumption efficiency. based neural network, Relevant Vector Machine (RVM), Feed-
A model for estimating the energy use and fuel consumption Forward Neural Network (FFNN), and Multiple Regression
was proposed by [23]. The authors exploited Automatic Iden- Model (MRM).
tification System (AIS) data and some technical information The authors in [28] used two datasets for predicting the fuel
about cruise ships, including the service speed, total power, consumption. With regards to the first dataset, the authors
and number of engines. A multivariate regression model was exploited the following set of features: ship shaft, speed,
trained. average draft, trims, current speed, current direction, wind
Reference [12] trained a multiple regression model used speed knots, wind direction, wave height, and wave direction.
for fuel consumption prediction. The authors used a limited In terms of the second dataset, the authors used the following
feature set consisting of ship average speed, sailed distance, feature set: ship shaft, speed, average draft, and trim. Finally,
wind speed in knots, and displacement. the authors used Back-Propagation Neural Network (BPNN)
Similarly, the authors in [24] used a multiple linear and Gaussian Process Regression (GPR) techniques.
regression model, where the amount of ship fuel consumed (in Reference [29] used voyage, weather, and sea data
litres) was designated as the output dependent variable, while for predicting fuel consumption. The authors trained a
factors such as the travelled distance in Nautical Mile (nm), LASSO regression model and compared it with ANNs, SVR,
travelled hours (HRS), Ship speed (V), Deadweight in metric and Gaussian Processes (GP). Findings showed that the
tonnes (DWT) and Wind Speed (W) in knots were designated proposed LASSO-based method outperforms other traditional
as the input (independent) variables. methods.

132578 VOLUME 11, 2023

L. Ilias et al.: Multitask Learning Framework for Predicting Ship Fuel Oil Consumption

A straightforward approach was also proposed by [30]. model, the authors introduced a speed optimization model
The authors trained traditional machine learning algorithms, between two ports.
including ANN, SVR, LASSO, and RF. They also used The authors in [37] trained an ANN and polynomial regres-
bayesian optimization for tuning the hyperparameters. The sion models to predict ship’s power and fuel consumption. The
authors used as features the relative wind direction, main authors used data from two voyages and exploited a limited
engine speed, fore and aft draft, and wind speed (knots). feature set, including the water depth, cargo conditions, and
In [31], the authors trained Gradient Boosting Regression more.
(GBR), Random Forest (RF), BP Network (BP), LR, and K- Similarly, the authors in [38] trained an ANN consisting
Nearest Neighbor Regression (kNN) to estimate the energy of one hidden layer with 10 units. The authors exploited
consumption of ships in port. 15 features were exploited voyage report data and used the following features as input to
including inherent ship features and external port features. the deep neural network: sailing speed (knots), displacement
Findings showed that net tonnage, deadweight tonnage, actual (MT), trim (m), wave height (m), wave direction, wind
weight, and efficiency of facilities are the top four features for force (Beaufort scale number), wind direction, and sea water
predicting the energy consumption of ships. temperature (◦ C). The output of the network was the fuel
A different approach was proposed by [32], where the consumption rate.
authors exploited the engine temperature as feature for the In [39], the authors introduced methods for predicting
estimation of fuel consumption for the first time and trained energy efficiency and addressing the optimal energy efficiency
an LSTM neural network. They compared the LSTM with route planning challenge. Specifically, for the energy
traditional machine learning algorithms, including linear efficiency prediction model, they utilized an ANN with a
regression, and ANNs and stated that the LSTM outperformed single hidden layer. For the optimal energy route planning
these approaches. Finally, the developed models have been issue, they presented an enhanced Ant Colony Algorithm.
implemented in optimisation of the engine speed to minimize Reference [14] introduced methods for estimating the
the total fuel consumption and the total cost of the whole required shaft power or main engine fuel consumption of a
voyage. Specifically, the authors employed the Reduced Space container ship sailing under varied conditions. The authors
Searching Algorithm for solving the optimization problems. exploited data acquired from the operation of a container
The study in [33] used in-service data collected from a ship. This dataset consists of 14 features, including longitude,
13,000 TEU class container ship to predict fuel consumption. latitude, draft aft, speed over ground, speed through water,
The authors used 11 features, including Speed over Ground and many more. Next, the authors applied data preprocessing
(SOG), speed through water, trim, displacement, etc. to train techniques, including time-series filtering, statistical outlier
machine learning algorithms. They trained a Multiple Linear detection, smoothing, etc. Then, data quality control was
Regression model and an ANN and showed that ANNs yielded performed. After that, the authors extracted a set of four
the best performance. features and applied feature selection approaches. Finally, the
The authors in [34] used data of two real-world voyages authors trained an ANN consisting of two hidden layers.
from bulk carrier and introduced a hybrid model for predicting
fuel consumption. Specifically, the introduced hybrid model B. BACKGROUND
is based on stacking theory consisting of two-level layers. 1) BIDIRECTIONAL LSTMS (BILSTMS)
The first layer is the base model, consisting of extremely Recurrent Neural Networks cannot capture the long-distance
randomized trees (ET), RF, and XGBoost (XGB). The second- dependencies effectively. To address this issue, Long Short-
level layer is a meta-model consisting of multiple linear Term Memory (LSTM) [40] neural models were proposed.
regression (MLR). Finally, the authors presented a new method An LSTM cell consists of three gates, namely the forget gate,
based on the developed hybrid model in combination with the the input gate, and the output gate. The structure of the LSTM
enumeration method to optimize the fuel consumption from unit is illustrated in Fig. 1 and is given by the equations below:
the perspective of trim adjustment. • The ft is the forget gate:
A similar approach was proposed by [35], where the authors
ft = σ Wf xt + Uf ht−1 + bf


used sensor data collected from an ocean-going container (1)
ship. The first-level layer is composed of multiple base- • The it is the input gate:
models, namely ET, RF, and XGB, while the second-level
layer is a meta model, i.e., multiple linear regression (MLR). it = σ (Wi xt + Ui ht−1 + bi ) (2)
The authors used nine features: GPS speed, mean draft, • The c˜t represents the candidate memory cell status at the
wave direction and height, wind direction, wind speed in current time-step.
knots, etc.
In [36], the authors introduced a two-stage fuel consumption c˜t = tanh (Wc xt + Ui ht−1 + bc ) (3)
prediction and reduction model. At the first stage, they trained • The ct represents the state value of the current time-step
a random forest regression model using 11 features, including in memory cell and is calculated as follows:
weather conditions, sea conditions, wind force, wave height,
and many more. Next, based on the random forest regression ct = it ⊙ c˜t + ft ⊙ ct−1 (4)

VOLUME 11, 2023 132579

 L. Ilias et al.: Multitask Learning Framework for Predicting Ship Fuel Oil Consumption

• The ot is the output gate:

ot = σ (Wo xt + Uo ht−1 + bo ) (5)

• ht indicates the hidden layer state at time t:

ht = ot ⊙ tanh (ct ) (6)

FIGURE 3. Scaled dot-product attention.

FIGURE 1. LSTM model.

a: SCALED DOT-PRODUCT ATTENTION
However, the typical LSTM layer cannot capture effectively A scaled dot-product attention can be described as mapping a
the contextual information. LSTM can only process sequence query and a set of key-value pairs to an output. As illustrated
from one direction. For this reason, research works utilize the in Fig. 3, the input consists of queries (Q), keys (K), and
bidirectional LSTM consisting of a forward LSTM layer and values (V). Specifically, queries and keys have a dimension
a backward LSTM layer. A BiLSTM layer is illustrated in of dk , while values have a dimension of dv . Firstly, the
Fig. 2. The equations governing the internal mechanism of a dot products of the query with all the keys are computed.
BiLSTM layer are presented as follows: Next, a scale operation is√applied, where the result of the
−
→ −−→ dot product is divided by dk . After that, a mask is applied
ht = LSTM (xt , ht−1 ) (7) optionally. Finally, a softmax function is applied for obtaining
←
− ←−− the weights on the values. It is worth noting that the scale
ht = LSTM (xt , ht+1 ) (8)
−
→ ← − operation is important, since the dot products grow large in
Ht = ht || ht , (9) magnitude, pushing the softmax function into regions where
−−→ it has extremely small gradients.
where xt denotes the input of time t, ht−1 denotes the The scaled dot-product attention can be calculated as
←−−
output of the forward hidden unit at time t − 1, while ht+1 follows:
indicates the output of the backward hidden unit. || denotes
QK T
 
the concatenation operation. Attention(Q, K , V ) = softmax √ V (10)
Dk

3) MULTIHEAD ATTENTION
To address the issue of performing one single attention, the
work in [41] introduces the multi-head attention mechanism,
which is illustrated in Fig. 4. Specifically, the multi-head
attention mechanism captures information from different
subspaces and strengthens the feature discrimination by
converting the original query matrix Q, key matrix K , and
value matrix V into H submatrices of the same size as
described in the equations presented below:
Q
Q = QWi ,
 i

.

FIGURE 2. BiLSTM model.
.

.



K i = KWiK , , (11)
2) MULTIHEAD ATTENTION MECHANISM ..


.


In this section, the multihead attention mechanism introduced 


V = VWiV ,
 i
in [41], is going to be described.

132580 VOLUME 11, 2023

L. Ilias et al.: Multitask Learning Framework for Predicting Ship Fuel Oil Consumption

where Qi ∈ Rn×dq , K i ∈ Rn×dk , and V i ∈ Rn×dv represent to the air pollution. Therefore, the prediction of both the main
the ith subspaces of Q, K , and V respectively. Usually, and auxiliary engine fuel consumption is crucial.
dq = dk = dmodel dmodel
H and dv = H . Therefore, the present study is significantly different from
the state-of-the-art approaches, since the authors (a) introduce
approaches for predicting both the main and auxiliary engine
fuel oil consumption, (b) exploit for the first time the multihead
self-attention, (c) introduce a multitask learning framework
which jointly learns to predict the main and auxiliary engine
fuel oil consumption, (d) introduce a new loss function in
the task of main and auxiliary engine fuel oil consumption
which mitigates the effect of ‘‘mimicking’’, and (e) evaluate
the approaches on three publicly available datasets with a great
number of features.

III. DATASET & TASK

A publicly available dataset [42] is used, which includes
sensor data from three different fishing ships for a period
of one month. This data has been used in Work Package
FIGURE 4. Multi-head attention. (WP) 3 of DataBio project, which has received funding from
Next, scaled dot-product attention operations are performed the European Union’s Horizon 2020 research and innovation
on H subspaces in parallel as follows: programme under grant agreement No 732064. Also, this data
is used in the context of the VesselAI project. The authors
head1 = Attention(Q1 , K 1 , V 1 ),

 resample the dataset into 5 minute bins and sum the values of
.

. the timestamps falling into a bin. A lag variable accounting

.


for 10 is used.

headi = Attention(Qi , K i , V i ), (12) Given a feature set, the tasks are to predict: (i) main
.

.. engine fuel oil consumption and (ii) auxiliary engine fuel oil




consumption. Table 2 describes the features used for the task


headH = Attention(QH , K H , V H ),

of main engine fuel oil consumption, while Table 3 reports the
Finally, the results learned by the multi-head attention are features used in terms of the task of the auxiliary engine fuel
concatenated as output and projected to dimensionality dmodel . oil consumption. As a preprocessing step, constant columns
Formally: are removed. Therefore, in these Tables, the features used per
dataset are mentioned.
p = MHA(Q, K , V ) = Concat(head1 , head2 , · · · , headH )W 0 ,
(13) IV. METHODOLOGY
In this section, the proposed models for predicting the
where W0∈ RHdv ×dmodel .
main engine and auxiliary engine fuel oil consumption
In case Q = K = V , then this attention can be called as
are described. Specifically, Section IV-A describes the
multi-head self-attention.
Single-Task Learning (STL) models that utilize either the
main engine Fuel Oil Consumption task or the auxiliary
C. RELATED WORK REVIEW FINDINGS
engine Fuel Oil Consumption task as the sole optimization
From the research works mentioned above, it is evident that
objective. Section IV-B describes the proposed architectures
existing research initiatives rely on the feature extraction
for jointly learning to predict main and auxiliary engine fuel
and train of shallow machine learning algorithms. Only
oil consumption. Finally, in IV-C the authors describe the
few research works have experimented with deep learning
proposed loss function used in the experiments.
approaches. However, these approaches introduce simple
architectures. For instance, the authors of some research works
A. SINGLE-TASK LEARNING
have employed ANNs and BiLSTMs instead of transformers,
which have achieved state-of-the-art results in many domains. The proposed STL architecture is illustrated in Fig. 5. Below,
Additionally, all the research works minimize the mean the layers of the proposed architecture are mentioned.
square error loss function without taking into consideration
the effect of ‘‘mimicking’’. Finally, the existing research 1) INPUT LAYER
initiatives mainly predict the main engine fuel oil consumption Let x ∈ Rn×T be the input representation, where n denotes the
neglecting the auxiliary engine. However, the auxiliary lag variable and is equal to 10, while T denotes the number of
machinery operates as a support of the main propulsion features. The dimension corresponding to the batch size is not
engines, while at the same time the auxiliary engines contribute mentioned for the sake of simplicity.

VOLUME 11, 2023 132581

 L. Ilias et al.: Multitask Learning Framework for Predicting Ship Fuel Oil Consumption

TABLE 2. Description of features - main engine.

In terms of the prediction of the main engine fuel oil where W Q ∈ R2l×Dq , W K ∈ R2l×Dk , and W V ∈ R2l×DV are
consumption, the authors use the feature set described in learnable parameters. As mentioned in [43], the authors set
Table 2. Dq = Dk = Dv = 2l. The self-attention mechanism can be
Regarding the prediction of the auxiliary engine fuel oil calculated as follows:
consumption, the authors exploit the feature set reported in 
QK T

Table 3. Attention(Q, K , V ) = softmax √ V (16)
Dk
2) BILSTM LAYER As an improved self-attention mechanism, MultiHead self-
x is passed through a BiLSTM layer. A BiLSTM consists of attention divides self-attention into h heads to learn the
−
→ different levels of long-term information in the input sequence.
two LSTMs, a forward LSTM f which processes the input
←
− The equation for calculating the attention of the i-th head is
sequence from left to right, and a backward LSTM f , which
processes the input sequence from right to left. Formally, described below:
−
→ ← − Q
headi = Attention(QWi , KWiK , VWiV ), (17)
hi = [ hi ; hi ], (14)
−
→ ← − Q
where hi , hi ∈ Rl and hi ∈ Rn×2l , where l denotes the where Wi ∈ R2l×dq , WiK ∈ R2l×dk , WiV ∈ R2l×dv . dq =
hidden dimensionality of the BiLSTM. dk = dv = 2lh
Finally, the results learned by the multi-head attention are
3) MULTIHEAD SELF-ATTENTION LAYER concatenated as output and projected to dimensionality d0 .
The authors exploit a MultiHead Self-Attention layer Formally:
introduced by [41]. Specifically, hi is transformed into a Query
p = MHA(Q, K , V ) = Concat(head1 , head2 , · · · , headh )W 0 ,
Q ∈ Rn×Dq , Key K ∈ Rn×Dk , and Value V ∈ Rn×Dv matrix
as described via the equations below: (18)

Q = QW Q , K = KW K , V = VW V , (15) where W 0 ∈ R2l×d0 . d0 = 2l.

132582 VOLUME 11, 2023

L. Ilias et al.: Multitask Learning Framework for Predicting Ship Fuel Oil Consumption

TABLE 3. Description of features - auxiliary engine.

Next, p is passed through a Global Average Pooling layer. propellers, shafting, steering gear, and deck cranes, or support
Let the output of the Global Average Pooling layer be z ∈ R. ship services like ballast water arrangements and sewage
systems. Additionally, multitask learning has been proved
4) OUTPUT LAYER to be effective in many domains [16], [17], [18], including
Finally, a dense layer consisting of 1 unit is used, in order to both related and unrelated tasks [19].
get the final prediction. In this section, two deep learning architectures based on
multi-task learning are introduced [44]. Each architecture
consists of two tasks, namely the main task and the auxiliary
one. The main objective is to explore whether the auxiliary
task helps the main task in increasing its performance. The
main task constitutes the task of the main engine fuel oil
consumption, while the auxiliary task constitutes the task of
the auxiliary engine fuel oil consumption. The introduced
architectures are trained on the two tasks and updated at the
same time with a joint loss:

L = (1 − α) LMEFOC + αLAEFOC , (19)

where LMEFOC denotes the loss of the main engine fuel oil
consumption, LAEFOC indicates the loss of the auxiliary engine
fuel oil consumption, and α is a parameter denoting the
importance the authors place to each task.

1) MTL-SIMPLE
In this architecture, the authors merge the features reported
in Tables 2 and 3 into one feature vector. The merging is
based on the correlation between the main engine and auxiliary
FIGURE 5. The proposed STL model. engines features, as the auxiliary engines, among others, are in
support of the main engine [11]. A lag variable of 10 is used.
Let x ∈ Rn×T be the input representation. n denotes the lag
B. MULTI-TASK LEARNING variable, while T denotes the number of features. As illustrated
According to [11], the auxiliary machinery operates as a in Fig. 6, the input representation is passed through a shared
support of the main propulsion engines. Auxiliary engine is BiLSTM layer as described in Eq. 14. Let z ∈ Rn×2h be
correlated with main engine as it can also assist the main the output of the BiLSTM layer, where h denotes the hidden
propulsion engines by incorporating heat exchangers and dimensionality of the BiLSTM. Next, z is passed through a
compressed air, aid in ship and cargo handling through MultiHead Self-Attention layer as described via the Equations.

VOLUME 11, 2023 132583

 L. Ilias et al.: Multitask Learning Framework for Predicting Ship Fuel Oil Consumption

FIGURE 6. The proposed MTL-simple model.

Let p ∈ Rn×2h be the output the MultiHead Self-Attention minimizing the MSE loss, become often sensitive to noise.
layer. Next, the authors pass p through a global average pooling This might result into the problem of predicting previously
layer and obtain s ∈ R2h . Finally, s is passed through two dense seen values (usually the last seen observation in the time
layers, where each dense layer consists of one unit, which give series), rather than making predictions based on long-term
the final prediction per task. extracted patterns. To tackle this limitation, the authors
in [15] introduce a regularization term for mitigating to
2) MTL-DOUBLE ENCODERS some degree the effect of ‘‘mimicking’’. For this reason,
The introduced architecture is illustrated in Fig. 7. Similar the following loss function is proposed for a sequence of n
to the MTL-Simple architecture, the authors merge the time-steps.
features reported in Tables 2 and 3 into one feature vector. n n
A lag variable of 10 is used. Let x ∈ Rn×T be the input
X X
L= (zi − ẑi )2 + λ [(zi − zi−1 )(zi − ẑi )]2 , (20)
representation. First, the authors pass x through a shared i=1 i=1
BiLSTM layer, which is updated by both tasks during training.
Let z ∈ Rn×2h be the output of the BiLSTM layer, where h where λ is a parameter used for controlling the importance of
denotes the hidden dimensionality of the BiLSTM. the regularization term, i.e., how much penalty needs to be
• Primary Task Prediction: This is a task-specific branch imposed to alleviate ‘‘mimicking’’.
pertinent to the primary task. Specifically, z is passed The loss function described in Eq. 20 is adopted in STL and
through a task-specific BiLSTM, a MultiHead Self- MTL frameworks. To be more precise, in terms of the MTL
Attention layer, a Global Average Pooling layer, and a setting, this specific loss function is exploited regarding both
Dense layer consisting of one unit which gives the final LMEFOC and LAEFOC (see Eq. 19).
output.
• Auxiliary Task Prediction: Here, the authors describe V. EXPERIMENTS
the task-specific branch related to the prediction of A. BASELINES
the auxiliary task. Specifically, z is passed through a The introduced approaches are compared with shallow
MultiHead Self-Attention layer followed by a global machine learning algorithms, namely AdaBoostRegressor,
average pooling layer. Finally, a dense layer consisting BaggingRegressor, ExtraTreesRegressor, GradientBoostin-
of one unit is used, which gives the final output, i.e., gRegressor, and RandomForestRegressor.
auxiliary engine fuel oil consumption.
B. EXPERIMENTAL SETUP
C. LOSS FUNCTION The authors scale the features and the output variable to a [0,1]
In [15], the authors define the problem of ‘‘mimicking’’ scale during training. After training, a simple post-processing
in time series forecasting. Specifically, neural networks step is applied where the predicted values are scaled to the

132584 VOLUME 11, 2023

L. Ilias et al.: Multitask Learning Framework for Predicting Ship Fuel Oil Consumption

FIGURE 7. The proposed MTL-double encoders model. This model consists of a

shared BiLSTM layer and two task-specific branches. The shared BiLSTM layer is
updated by both tasks, while each task-specific branch is updated by the
corresponding task.

actual range of values. α of Eq. 19 is set equal to 0.1. λ of • Coefficient of determination:

Eq. 20 is set equal to 1. For STL and MTL-Simple models, Pn
2
the hidden size of the BiLSTM layer is h = 150. For the 2 t=1 yt − yˆt
R =1− Pn (21)
t=1 (yt − y)
2
MTL-Double Encoders model, the hidden size of the shared
BiLSTM layer is h = 150, while the hidden size of the • Mean Bias Error:
BiLSTM corresponding to the task-specific branch of the n
primary task is h = 200. The authors split the dataset into 1X
MBE = (yt − yˆt ) (22)
a train and test set. Additionally, the authors divide the train n
t=1
set into a train and validation set. A batch size of 64 is used.
• Root Mean Squared Error:
The authors use EarlyStopping, where training is stopped if
v
the validation loss has stopped decreasing for 10 consecutive u n
u1 X
epochs. Adam optimizer [45] is used for all the experiments. RMSE = t (yt − ŷt )2 (23)
The models are trained for a maximum of 200 epochs. n
i=1
The authors repeat the experiments 20 times and report the
• Mean Absolute Error:
mean results. The authors use the t-test for significance Pn
testing. The authors use PyTorch [46] for performing the i=1 |yt − ŷt |
MAE = (24)
experiments. All experiments are trained on a single Tesla n
P100-PCIE-16GB GPU.
VI. RESULTS
The results of the proposed approaches mentioned in
C. EVALUATION METRICS Section IV are reported in Tables 4 and 5.
The authors utilize the metrics mentioned below for evaluating Regarding the Dataset 1, the authors observe in Table 4 that
the results of the introduced approaches. Specifically, yt MTL-Double Encoders constitutes the best performing model
refers to the real value of the fuel oil consumption, achieving the best results across all metrics for predicting
while ŷt refers to the predicted value of the fuel oil the main engine fuel oil consumption. Specifically, the best
consumption. y denotes the mean of the real values of fuel oil performing model improves R2 score by 0.22-17.06%, RMSE
consumption. presents an improvement of 0.05-2.37, while MAE is also

VOLUME 11, 2023 132585

 L. Ilias et al.: Multitask Learning Framework for Predicting Ship Fuel Oil Consumption

TABLE 4. Performance comparison among traditional ML algorithms and proposed STL and MTL models on primary prediction task (prediction of main
engine fuel oil consumption). Results for STL and MTL approaches are averaged across 20 runs. Best results per evaluation metric are underlined. † indicates
significant improvement over STL and MTL-Simple (t-test, p-value<0.05).

TABLE 5. Performance comparison among traditional ML algorithms and proposed STL and MTL models on auxiliary prediction task (prediction of auxiliary
engine fuel oil consumption). Results for STL and MTL approaches are averaged across 20 runs. Best results per evaluation metric are underlined.

improved by 0.02-3.15. The authors observe that the AdaBoost by 3.19%, in MBE by 0.10, in RMSE by 0.07, and in MAE
Regressor obtains the worst evaluation results yielding an by 0.08.
R2 score of 77.91%. MTL-Double Encoders outperforms In terms of the Dataset 2, the authors observe in Table 4 that
significantly both MTL-Simple and STL in terms of R2 score MTL-Double Encoders constitutes the best performing model
by 0.23% (p-value=9.26e-14) and 0.32% (p-value=1.50e- surpassing the rest of the approaches, i.e., traditional ML algo-
07) respectively. The best performing model obtains better rithms and introduced approaches, in R2 score by 0.06-6.20%
RMSE than MTL simple by 0.05 (p-value=3.50e-05) and and in RMSE by 0.01-1.21. It achieves a MAE of 1.02 which
STL by 0.07 (p-value=1.50e-07). In terms of MBE, the is equal with the one obtained by ExtraTreesRegressor, while
authors observe that the best performing model achieves it outperforms the rest approaches by 0.06-2.05. Compared
the lowest MBE accounting for -0.02 with an improvement with the introduced approaches, MTL-Double Encoders
over STL and MTL-Simple of 0.05 (p-value=1.10e-05) and outperforms STL and MTL-Simple in R2 score by 0.57% (p-
0.12 (p-value=0.00278) respectively. Observing Table 5, the value=1.99e-12) and 0.42% (p-value=6.51e-09) respectively,
authors observe that the proposed STL model outperforms in MBE by 0.17 (p-value=7.71e-12) and 0.15 (p-value=3.97e-
the introduced MTL approaches. This can be justified by 11) respectively, in RMSE by 0.13 (p-value=2.81e-12) and
the fact that the authors have set α of Eq. 19 equal to 0.10 (p-value=7.57e-09) respectively, and in MAE by 0.08
0.1, placing in this way importance to the task of the (p-value=1.89e-08) and 0.10 (p-value=1.14e-09) respectively.
prediction of main engine fuel oil consumption. As one As one can observe in Table 5, the proposed STL model
can observe in Table 5, the proposed STL model surpasses attains an R2 score of 94.37% outperforming the traditional
the performance of traditional ML algorithms in R2 score ML algorithms by 0.57-4.45%. Additionally, STL improves
by 0.15-4.27% and in RMSE by 0.01-0.10. Although the RMSE over traditional ML algorithms by 0.04-0.14 and MAE
differences in performance are limited, the authors believe by 0.02-0.19.
that even a small improvement can make the difference in With regards to the Dataset 3, the authors observe in Table 4
this field. In terms of the MTL approaches, although the that MTL-Double Encoders constitutes the best performing
authors do not place enough importance in this task, the model obtaining an R2 score of 99.45%, a MBE of 0.16,
authors observe that their approaches yield competitive results an RMSE of 0.99, and a MAE of 0.36. Specifically, MTL-
with traditional ML algorithms. Specifically, the authors Double Encoders outperforms MTL-Simple in R2 score by
observe that MTL-Simple outperforms AdaBoostRegressor 0.17% (p-value=7.89e-09), in MBE by 0.23 (p-value=7.03e-
in terms of R2 , MBE, RMSE, and MAE. For instance, 15), in RMSE by 0.14 (p-value=3.11e-09), and in MAE by
MTL-Simple outperforms AdaBoostRegressor in R2 score 0.22 (p-value=7.96e-16). Similarly, MTL-Double Encoders

132586 VOLUME 11, 2023

L. Ilias et al.: Multitask Learning Framework for Predicting Ship Fuel Oil Consumption

outperforms STL in R2 score by 0.24% (p-value=4.89e-08), example, an accurate fuel consumption prediction can inform
in MBE by 0.28 (p-value=2.09e-13), in RMSE by 0.19 (p- operators about the most fuel-efficient trim for the vessel
value=4.33e-09), and in MAE by 0.26 (p-value=8.27e-16). under different conditions. Moreover, by correlating accurate
In comparison with the traditional ML regressors, MTL- fuel consumption predictions with varying ballast levels,
Double Encoders yields a better R2 score by 0.32-9.37%, operators can identify the optimal amount and distribution of
improves RMSE by 0.24-3.18, and MAE by 0.21-3.49. As one ballast. Additionally uneven or suboptimal cargo distribution
can easily observe in Table 5, the proposed STL model can can result in increased resistance and, consequently, higher
predict the auxiliary engine fuel oil consumption attaining fuel consumption. Accurate predictions of fuel consumption,
an R2 score of 90.79% which is better than 2.22-20.68% in based on different cargo distributions, can guide operators in
comparison with the traditional ML algorithms. Regarding the distributing cargo more efficiently, optimizing the underwater
proposed approaches in the MTL framework, one can observe part of the ship and its hydrodynamics. Lastly, it can contribute
that they achieve competitive results. Specifically, both MTL to proactive maintenance and performance monitoring of the
approaches outperform three ML algorithms in R2 score. vessel’s engines as well as their optimal tuning.
Overall, one can observe that MTL-Double Encoders However, this study comes with some limitations. Specifi-
constitutes the best performing model across all datasets. cally, the authors did not apply hyperparameter tuning, which
We speculate that this is attributable to the fact that this often leads to a performance improvement. In addition, the
architecture consists of both shared and task-specific branches. authors did not apply explainability techniques for rendering
the proposed approaches explainable. Finally, the authors
VII. CONCLUSION & FUTURE WORK experimented only with fishing ships and did not test other
In this paper, the authors present the first study exploiting ship types.
transformer-based approaches for predicting both the main and In the future, the authors aim to propose deep learning
auxiliary engine fuel oil consumption. Specifically, the authors approaches for the task of route optimization. Also, the
introduce both single-task and multi-task learning models. prediction of CO2 emissions is one of their future plans.
In terms of the single-task learning setting, the proposed Additionally, the authors aim to apply the proposed models to
models consist of BiLSTMs and MultiHead Self-Attention other ship types. Finally, the authors aim to contribute to this
layers. The authors further use a multi-task learning framework field by proposing explainable deep neural networks.
to jointly model the main engine fuel oil consumption and
auxiliary engine fuel oil consumption as an auxiliary task. REFERENCES
In order to address the phenomenon of ‘‘mimicking’’ in time- [1] H. N. Psaraftis and C. A. Kontovas, ‘‘Speed models for energy-efficient
maritime transportation: A taxonomy and survey,’’ Transp. Res. C,
series forecasting which is a consequence of minimizing the Emerg. Technol., vol. 26, pp. 331–351, Jan. 2013. [Online]. Available:
MSE loss, the authors add a regularization term in the loss https://www.sciencedirect.com/science/article/pii/S0968090X12001246
function. The authors evaluate the proposed approaches in [2] C. Bagoulla and P. Guillotreau, ‘‘Maritime transport in the French
economy and its impact on air pollution: An input–output analysis,’’
three publicly available datasets, which include sensor data Mar. Policy, vol. 116, Jun. 2020, Art. no. 103818. [Online]. Available:
from fishing vessels. Findings show that the proposed MTL https://www.sciencedirect.com/science/article/pii/S0308597X19305408
approaches outperform significantly the traditional ones. [3] P. P. Vinayak, C. S. K. Prabu, N. Vishwanath, and S. O. Prakash, ‘‘Numerical
simulation of ship navigation in rough seas based on ECMWF data,’’
The significantly increased performance and accuracy of Brodogradnja, vol. 72, no. 1, pp. 19–58, Mar. 2021.
the introduced MTL approach can result to a number of [4] A. Farkas, N. Degiuli, I. Martić, and A. Mikulić, ‘‘Benefits of slow
measures and technologies that can contribute to the overall steaming in realistic sailing conditions along different sailing routes,’’
Ocean Eng., vol. 275, May 2023, Art. no. 114143. [Online]. Available:
fuel oil consumption and thus the operating costs of shipping https://www.sciencedirect.com/science/article/pii/S0029801823005279
companies. By predicting fuel oil consumption through the [5] N. Degiuli, I. Martić, A. Farkas, and I. Gospić, ‘‘The impact of
feature sets of the main and auxiliary engine, more insights slow steaming on reducing CO2 emissions in the Mediterranean Sea,’’
Energy Rep., vol. 7, pp. 8131–8141, Nov. 2021. [Online]. Available:
can be provided for the ship performance with data based https://www.sciencedirect.com/science/article/pii/S235248472100144X
on real operating conditions that can further impact different [6] A. Farkas, N. Degiuli, I. Martić, and C. G. Grlj, ‘‘Is slow steaming a viable
sectors and activities of the maritime industry. For instance, option to meet the novel energy efficiency requirements for containerships?’’
a better fuel consumption prediction model can further J. Cleaner Prod., vol. 374, Nov. 2022, Art. no. 133915. [Online]. Available:
https://www.sciencedirect.com/science/article/pii/S0959652622034874
improve the accuracy and performance fuel prediction services [7] C. Dere and C. Deniz, ‘‘Load optimization of central cooling sys-
of commercial fleet monitoring and stability management tem pumps of a container ship for the slow steaming conditions
software that has been tailored to specific clients/ships (main to enhance the energy efficiency,’’ J. Cleaner Prod., vol. 222,
pp. 206–217, Jun. 2019. [Online]. Available: https://www.sciencedirect.
and auxiliary engines features) and thus enables them for better com/science/article/pii/S0959652619307139
weather routing optimization planning. Moreover, they can [8] C. G. Grlj, N. Degiuli, Ž. Tuković, A. Farkas, and I. Martić,
directly contribute to supporting the decision-making process ‘‘The effect of loading conditions and ship speed on the
wind and air resistance of a containership,’’ Ocean Eng.,
for ship energy design systems for feeding ship energy systems vol. 273, Apr. 2023, Art. no. 113991. [Online]. Available:
simulations and optimization. Furthermore, it can provide https://www.sciencedirect.com/science/article/pii/S002980182300375X
guidance and insights for adjusting a vessel’s trim, ballast, [9] M. Kalajdžić, M. Vasilev, and N. Momčilović, ‘‘Power reduction
considerations for bulk carriers with respect to novel energy efficiency
and cargo distribution in order to improve its hydrodynamics regulations,’’ Brodogradnja: Teorija i praksa brodogradnje i pomorske
and reduce fuel consumption usage and optimal trim. For tehnike, vol. 73, no. 2, pp. 79–92, 2022.

VOLUME 11, 2023 132587

 L. Ilias et al.: Multitask Learning Framework for Predicting Ship Fuel Oil Consumption

[10] C. Sui, P. de Vos, D. Stapersma, K. Visser, and Y. Ding, ‘‘Fuel consumption [28] Z. Hu, Y. Jin, Q. Hu, S. Sen, T. Zhou, and M. T. Osman, ‘‘Prediction of fuel
and emissions of ocean-going cargo ship with hybrid propulsion and consumption for enroute ship based on machine learning,’’ IEEE Access,
different fuels over voyage,’’ J. Mar. Sci. Eng., vol. 8, no. 8, p. 588, vol. 7, pp. 119497–119505, 2019.
Aug. 2020. [Online]. Available: https://www.mdpi.com/2077-1312/8/8/588 [29] S. Wang, B. Ji, J. Zhao, W. Liu, and T. Xu, ‘‘Predicting ship
[11] A. F. Molland, ‘‘Chapter 6—Marine engines and auxiliary fuel consumption based on LASSO regression,’’ Transp. Res. D,
machinery,’’ in The Maritime Engineering Reference Book, Oxford: Transp. Environ., vol. 65, pp. 817–824, Dec. 2018. [Online]. Available:
Butterworth-Heinemann, 2008, pp. 344–482. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1361920917302109
https://www.sciencedirect.com/science/article/pii/B9780750689878000068 [30] T. Zhou, Q. Hu, Z. Hu, and R. Zhen, ‘‘An adaptive hyper
[12] D. Bocchetti, A. Lepore, B. Palumbo, and L. Vitiello, ‘‘A statistical control of parameter tuning model for ship fuel consumption prediction
the ship fuel consumption,’’ in Proc. Int. Conf. Design, Construct. Operation under complex maritime environments,’’ J. Ocean Eng. Sci.,
Passenger Ships, Roy. Inst. Nav. Architects, Nov. 2013, pp. 20–21. vol. 7, no. 3, pp. 255–263, Jun. 2022. [Online]. Available:
[13] M. Jeon, Y. Noh, Y. Shin, O.-K. Lim, I. Lee, and D. Cho, ‘‘Prediction of https://www.sciencedirect.com/science/article/pii/S2468013321000759
ship fuel consumption by using an artificial neural network,’’ J. Mech. Sci. [31] Y. Peng, H. Liu, X. Li, J. Huang, and W. Wang, ‘‘Machine learning method
Technol., vol. 32, no. 12, pp. 5785–5796, Dec. 2018. for energy consumption prediction of ships in port considering green ports,’’
[14] P. Karagiannidis and N. Themelis, ‘‘Data-driven modelling of J. Cleaner Prod., vol. 264, Aug. 2020, Art. no. 121564. [Online]. Available:
ship propulsion and the effect of data pre-processing on the https://www.sciencedirect.com/science/article/pii/S0959652620316115
prediction of ship fuel consumption and speed loss,’’ Ocean [32] Z. Yuan, J. Liu, Q. Zhang, Y. Liu, Y. Yuan, and Z. Li,
Eng., vol. 222, Feb. 2021, Art. no. 108616. [Online]. Available: ‘‘Prediction and optimisation of fuel consumption for inland ships
https://www.sciencedirect.com/science/article/pii/S0029801821000512 considering real-time status and environmental factors,’’ Ocean
[15] C. Kosma, G. Nikolentzos, N. Xu, and M. Vazirgiannis, ‘‘Time series Eng., vol. 221, Feb. 2021, Art. no. 108530. [Online]. Available:
forecasting models copy the past: How to mitigate,’’ in Proc. Artif. https://www.sciencedirect.com/science/article/pii/S0029801820314372
Neural Netw. Mach. Learn. (ICANN), E. Pimenidis, P. Angelov, C. Jayne, [33] Y.-R. Kim, M. Jung, and J.-B. Park, ‘‘Development of a fuel consumption
A. Papaleonidas, and M. Aydin, Eds. Cham, Switzerland: Springer, 2022, prediction model based on machine learning using ship in-service data,’’
pp. 366–378. J. Mar. Sci. Eng., vol. 9, no. 2, p. 137, Jan. 2021. [Online]. Available:
[16] L. Ilias and D. Askounis, ‘‘Explainable identification of dementia from https://www.mdpi.com/2077-1312/9/2/137
transcripts using transformer networks,’’ IEEE J. Biomed. Health Informat.,
[34] Z. Hu, T. Zhou, R. Zhen, Y. Jin, X. Li, and M. T. Osman, ‘‘A two-step strategy
vol. 26, no. 8, pp. 4153–4164, Aug. 2022.
for fuel consumption prediction and optimization of ocean-going ships,’’
[17] S. Rajamanickam, P. Mishra, H. Yannakoudakis, and E. Shutova, ‘‘Joint
Ocean Eng., vol. 249, Apr. 2022, Art. no. 110904. [Online]. Available:
modelling of emotion and abusive language detection,’’ in Proc. 58th
https://www.sciencedirect.com/science/article/pii/S0029801822003407
Annu. Meeting Assoc. Comput. Linguistics, 2020, pp. 4270–4279. [Online].
[35] Z. Hu, T. Zhou, M. T. Osman, X. Li, Y. Jin, and R. Zhen, ‘‘A novel hybrid
Available: https://aclanthology.org/2020.acl-main.394
fuel consumption prediction model for ocean-going container ships based
[18] M. Jin and N. Aletras, ‘‘Modeling the severity of complaints in social
on sensor data,’’ J. Mar. Sci. Eng., vol. 9, no. 4, p. 449, Apr. 2021. [Online].
media,’’ in Proc. Conf. North Amer. Chapter Assoc. Comput. Linguistics,
Available: https://www.mdpi.com/2077-1312/9/4/449
Human Lang. Technol., Jul. 2021, pp. 2264–2274. [Online]. Available:
https://aclanthology.org/2021.naacl-main.180 [36] R. Yan, S. Wang, and Y. Du, ‘‘Development of a two-stage
[19] B. R. Paredes, A. Argyriou, N. Berthouze, and M. Pontil, ‘‘Exploiting ship fuel consumption prediction and reduction model for
unrelated tasks in multi-task learning,’’ in Proc. 15th Int. Conf. Artif. a dry bulk ship,’’ Transp. Res. E, Logistics Transp. Rev.,
Intell. Statist., vol. 22, N. D. Lawrence and M. Girolami, Eds. La Palma, vol. 138, Jun. 2020, Art. no. 101930. [Online]. Available:
Canary Islands: PMLR, Apr. 2012, pp. 951–959. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1366554519308555
https://proceedings.mlr.press/v22/romera12.html [37] Y. B. A. Farag and A. I. Ölçer, ‘‘The development of a ship
[20] C. Gkerekos, I. Lazakis, and G. Theotokatos, ‘‘Machine learning performance model in varying operating conditions based on ANN
models for predicting ship main engine fuel oil consumption: and regression techniques,’’ Ocean Eng., vol. 198, Feb. 2020,
A comparative study,’’ Ocean Eng., vol. 188, Sep. 2019, Art. no. 106972. [Online]. Available: https://www.sciencedirect.
Art. no. 106282. [Online]. Available: https://www.sciencedirect. com/science/article/pii/S0029801820300536
com/science/article/pii/S0029801819304561 [38] Y. Du, Q. Meng, S. Wang, and H. Kuang, ‘‘Two-phase optimal
[21] E. B. Beşikçi, O. Arslan, O. Turan, and A. I. Ölçer, ‘‘An artificial neural solutions for ship speed and trim optimization over a voyage
network based decision support system for energy efficient ship operations,’’ using voyage report data,’’ Transp. Res. B, Methodol., vol. 122,
Comput. Oper. Res., vol. 66, pp. 393–401, Feb. 2016. [Online]. Available: pp. 88–114, Apr. 2019. [Online]. Available: https://www.sciencedirect.
https://www.sciencedirect.com/science/article/pii/S0305054815000842 com/science/article/pii/S0191261517305738
[22] J. P. Petersen, O. Winther, and D. J. Jacobsen, ‘‘A machine-learning [39] C. Zhang, D. Zhang, M. Zhang, and W. Mao, ‘‘Data-driven ship
approach to predict main energy consumption under realistic operational energy efficiency analysis and optimization model for route planning
conditions,’’ Ship Technol. Res., vol. 59, no. 1, pp. 64–72, Jan. 2012, doi: in ice-covered Arctic waters,’’ Ocean Eng., vol. 186, Aug. 2019,
10.1179/str.2012.59.1.007. Art. no. 106071. [Online]. Available: https://www.sciencedirect.
[23] M. Simonsen, H. Walnum, and S. Gössling, ‘‘Model for estimation of fuel com/science/article/pii/S0029801819302744
consumption of cruise ships,’’ Energies, vol. 11, no. 5, p. 1059, Apr. 2018. [40] S. Hochreiter and J. Schmidhuber, ‘‘Long short-term memory,’’
[Online]. Available: https://www.mdpi.com/1996-1073/11/5/1059 Neural Comput., vol. 9, no. 8, pp. 1735–1780, Nov. 1997, doi:
[24] K.-K. Kee, B.-Y. L. Simon, and K.-H. Y. Renco, ‘‘Prediction of ship fuel 10.1162/neco.1997.9.8.1735.
consumption and speed curve by using statistical method,’’ J. Comput. Sci. [41] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez,
Comput. Math., vol. 8, no. 2, pp. 19–24, Jun. 2018. L. U. Kaiser, and I. Polosukhin, ‘‘Attention is all you need,’’ in Advances in
[25] A. Lepore, M. S. dos Reis, B. Palumbo, R. Rendall, and Neural Information Processing Systems, vol. 30, I. Guyon, U. V. Luxburg,
C. Capezza, ‘‘A comparison of advanced regression techniques S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, Eds.
for predicting ship CO2 emissions,’’ Qual. Rel. Eng. Int., Red Hook, NY, USA: Curran Associates, 2017. [Online]. Available:
vol. 33, no. 6, pp. 1281–1292, Oct. 2017. [Online]. Available: https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053
https://onlinelibrary.wiley.com/doi/abs/10.1002/qre.2171 c1c4a845aa-Paper.pdf
[26] T. Uyanık, Ç. Karatuğ, and Y. Arslanoğlu, ‘‘Machine learning approach [42] P. Siltanen and Z. U. Arrue, ‘‘Sensor data from three different
to ship fuel consumption: A case of container vessel,’’ Transp. Res. D, fishing ships for a period of one month,’’ Zenodo, Version 1.0.0,
Transp. Environ., vol. 84, Jul. 2020, Art. no. 102389. [Online]. Available: Dec. 2019. [Online]. Available: https://zenodo.org/records/3563390, doi:
https://www.sciencedirect.com/science/article/pii/S1361920920305769 10.5281/zenodo.3563390.
[27] I. Panapakidis, V.-M. Sourtzi, and A. Dagoumas, ‘‘Forecasting the fuel [43] C. Chen, D. Han, and C.-C. Chang, ‘‘CAAN: Context-aware
consumption of passenger ships with a combination of shallow and deep attention network for visual question answering,’’ Pattern
learning,’’ Electronics, vol. 9, no. 5, p. 776, May 2020. [Online]. Available: Recognit., vol. 132, Dec. 2022, Art. no. 108980. [Online]. Available:
https://www.mdpi.com/2079-9292/9/5/776 https://www.sciencedirect.com/science/article/pii/S0031320322004605

132588 VOLUME 11, 2023

L. Ilias et al.: Multitask Learning Framework for Predicting Ship Fuel Oil Consumption

[44] R. Caruana, ‘‘Multitask learning,’’ Mach. Learn., vol. 28, no. 1, pp. 41–75, SPIROS MOUZAKITIS received the Ph.D. degree
1997. from the School of Electrical and Computer
[45] D. P. Kingma and J. Ba, ‘‘Adam: A method for stochastic optimization,’’ Engineering, National Technical University of
2014, arXiv:1412.6980. Athens, Athens, Greece, in 2009. He is currently a
[46] A. Paszke et al., ‘‘Pytorch: An imperative style, high-performance Senior Research Analyst with the Decision Support
deep learning library,’’ in Advances in Neural Information Systems Laboratory, National Technical University
Processing Systems, vol. 32, H. Wallach, H. Larochelle, of Athens. He has more than ten years of industry
A. Beygelzimer, F. d’Alché-Buc, E. Fox, and R. Garnett, Eds. Red experience in software engineering for banking, e-
Hook, NY, USA: Curran Associates, 2019. [Online]. Available: commerce systems, and energy suppliers. He was
https://proceedings.neurips.cc/paper/2019/file/bdbca288fee7f92f2bfa9f701
a Project Manager for 16 years in more than
2727740-Paper.pdf
20 projects (H2020, FP7, FP6, and FP5 research projects) related to big data,
open data, market/impact analysis in the context of ICT technologies, web
development, and enterprise interoperability. He has published in numerous
journals and presented his research at international conferences. His current
research interests include decision analysis in the field of decision support
LOUKAS ILIAS received the integrated master’s systems based on machine learning/deep learning, big and linked data
degree from the School of Electrical and Computer analytics, and optimization systems and algorithms.
Engineering (SECE), National Technical Univer-
sity of Athens (NTUA), Athens, Greece, in June
2020, where he is currently pursuing the Ph.D.
degree with the Decision Support Systems (DSS)
Laboratory, SECE. He has completed a Research
Internship with University College London (UCL),
London, U.K. He is a Researcher with the DSS
Laboratory, NTUA, where he is involved in EU-
funded research projects. He has published in numerous journals, including
IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS, IEEE TRANSACTIONS ON
COMPUTATIONAL SOCIAL SYSTEMS, Knowledge-Based Systems (Elsevier), Expert
Systems With Applications (Elsevier), Applied Soft Computing (Elsevier),
Computer Speech and Language (Elsevier), Online Social Networks and
Media (Elsevier), IEEE ACCESS, and Frontiers in Aging Neuroscience. His
research has also been accepted for presentation at international conferences,
including the IEEE-EMBS International Conference on Biomedical and
Health Informatics (BHI’22) and the IEEE International Conference on
Acoustics, Speech, and Signal Processing (ICASSP) 2023. His research
interests include speech processing, natural language processing, social media DIMITRIS ASKOUNIS for a number of years, was
analysis, and the detection of complex brain disorders. an Advisor with the Minister of Justice and the
Special Secretary of the Digital Convergence for
the introduction of information and communication
technologies in public administration. Since June
2019, he has been the President of the Information
PANAGIOTIS KAPSALIS received the Dipl.-Ing. Society SA, Kallithea, Greece. He is currently a
degree in computer and informatics engineering Professor with the School of Electrical and Com-
from the University of Patras, in 2016, and the M.Sc. puter Engineering, National Technical University
degree in applied data science from the Athens of Athens (NTUA), Athens, Greece, and the Deputy
University of Economics and Business, in 2017. Director of the Decision Support Systems Laboratory. He has over 25 years
He is currently pursuing the Ph.D. degree with of experience in decision support systems, intelligent information systems
the National Technical University of Athens. Since and manufacturing, e-business, e-government, open and linked data, big data
2017, he has been a Research Software Engineer in analytics, artificial intelligence (AI) algorithms, and the application of modern
EU funded projects. He has presented his research information technology (IT) techniques in the management of companies and
at international conferences. His research interests organizations. He was the Scientific Director of over 50 European research
include information systems, big data systems and data management, and projects in the above areas (FP7 and Horizon2020).
software engineering.

VOLUME 11, 2023 132589