Draft_Copy_Implementation of the Proposed Method
Draft_Copy_Implementation of the Proposed Method
Abstract.......................................................................................................2
Introduction.................................................................................................2
Literature Study...........................................................................................3
Implementation of the Proposed Method (Methodology)............................5
MODEL-1..................................................................................................5
Adding Temporal Analysis on Weight Factor..........................................6
MODEL-2..................................................................................................6
Proposed Algorithm.....................................................................................8
Model Parameters and Settings..............................................................11
Temperature Prediction (ForestFires)......................................................12
Results.......................................................................................................17
Comparative Results..............................................................................20
Conclusion.................................................................................................20
Pseudo Code..............................................................................................21
Reference..................................................................................................22
Abstract
This study presents a comparative analysis of two predictive modeling
approaches for time series forecasting: a Long Short-Term Memory (LSTM)
network and a Feedforward Neural Network (FNN). We begin by
preprocessing the dataset through normalization and sequence creation,
where sequences of data are prepared for model input. The LSTM model is
designed with a single hidden layer of 50 units and ReLU activation, and is
trained using mean squared error (MSE) as the loss function. In parallel,
an FNN model with a three-layer architecture, including 64 and 32 units in
the hidden layers and a single output unit, is trained under similar
conditions. Both models are evaluated on their predictive performance
using the MSE metric, with the results highlighting the comparative
effectiveness of each approach in time series forecasting. This analysis
provides insights into the strengths and limitations of LSTM and FNN
models, contributing to the understanding of their applicability in
predictive analytics.
Introduction
The rapid advancement of the Internet of Things (IoT) has ushered in
transformative changes across various industries, with precision
agriculture, or smart farming, standing out as a key beneficiary [1][2]. By
integrating IoT technologies, traditional farming practices have evolved
into more efficient and data-driven processes, significantly enhancing
productivity and resource management [3]. This shift has been
instrumental in optimizing operations such as soil moisture monitoring,
climate tracking, and precision irrigation, thereby boosting crop yields and
reducing waste.
Our research not only addresses a critical gap in the existing literature but
also offers practical solutions for improving the reliability of IoT systems in
agriculture. By enhancing anomaly detection capabilities, we contribute to
the broader goal of achieving more efficient and secure smart farming
practices. The novelty and Contribution of the proposed work is given
below:
Yazdinejad et al. (2021) review the security aspects of smart farming and
precision agriculture, discussing the different types of attacks, threats,
and countermeasures relevant to this field. Their work emphasizes the
need for ongoing research and development to address the evolving
security challenges faced by IoT applications in agriculture [6].
Mohammad et al. (2019) explore security weaknesses and attacks on IoT
applications more broadly, providing a comprehensive overview of the
types of threats that can affect various IoT systems. Their research
highlights the need for improved security protocols to protect against
these vulnerabilities [7].
Karim et al. (2017) propose LSTM fully convolutional networks for time
series classification, offering a robust method for analyzing temporal
patterns in sensor data. Their work is relevant for enhancing the detection
of temporal anomalies in IoT systems, particularly in the context of smart
farming [10]. Xu, He, and Li (2014) provide a comprehensive survey on
the deployment of IoT in industrial contexts. Their work details the various
industrial applications of IoT, emphasizing how these technologies
enhance operational efficiency, automate processes, and facilitate real-
time monitoring. The survey covers a wide range of industrial sectors,
illustrating the diverse applications and potential benefits of IoT
integration in industrial environments [11].
Liu et al. (2021) explore the transition from Industry 4.0 to Agriculture 4.0,
examining the current state of IoT applications in agriculture. They identify
enabling technologies, such as sensors and data analytics, that drive the
shift towards smarter agricultural practices. The paper also highlights key
research challenges, including the need for robust systems to handle large
volumes of data and ensure the reliability of IoT solutions in agricultural
settings [12].
∑ ❑ wij ( x j−x́ )2
δ 2i = j =1
N−1
It will help to resolve the issues of static sensor as well as low density
deployment.
MODEL-2
Model 2 expands on this approach by introducing a more detailed
breakdown of spatial and temporal correlations. It includes first-order
N
spatial correlation ( Si ), calculated as Si=¿ ∑ j=1 , j≠ i wij ( x j− x́ ), and first-order
temporal correlation ( T i ), defined as the difference between consecutive
time points, T i=x i (t)−x i (t−1). Additionally, second-order correlations are
introduced: second-order spatial correlation ( S(2)
i ) and second-order
temporal correlation ( T (2i ) ), capturing more complex patterns in the data.
These are combined into a second-order spatialtemporal correlation
equation: C i=α Si + β T i + γ S (2) (2)
i + δ T i , where α , β , γ , and δ are weighting factors.
i +δ T i ], integrates
The final anomaly detection equation, Ai= ( x i (t)− x́ ) ¿ γ S(2) (2)
1
where w ij = 2.
dij
First-Order Temporal Correlation:
T i=x i (t)−x i (t−1)
Second-Order Equation
In the below figure neural network settings are shown, including 100
training cycles, a learning rate of 0.005, and training on a CPU without
using a learned optimizer. The network architecture consists of an input
layer with 70 features, followed by two dense layers with 20 and 10
neurons, respectively, and a single output layer. This configuration
suggests a basic feedforward neural network setup for a supervised
learning task.
Proposed Algorithm
Algorithm 1 describes the process for developing and evaluating a
predictive LSTM model. The initial step involves importing essential
libraries such as numpy, 'pandas, 'tensorflow', andkeras, along with
modules for data scaling and performance evaluation. The dataset is
then loaded from a CSV file usingread_csv(file_path)`. In the data
preprocessing phase, features (' x ') and target variables (' y ) are
extracted from the dataset. The features are normalized
withStandardScaler()` to standardize the input values, which helps in
improving model performance. Sequences of data are then created using
a defined sequence length ('SEQ LENGTH'), which structures the data into
a format suitable for LSTM input.
For model development, the dataset is divided into training and testing
sets using 'train_test_split()', reserving 20% of the data for testing. An
LSTM model is then defined and compiled with the Adam optimizer and
mean squared error loss function, setting the stage for training. During the
training phase, the model is trained using model1.fit( ) with a specified
number of epochs (100) and a validation split of 10 % . This training process
adjusts the model's parameters to minimize the loss on the training data.
In the evaluation phase, the model's performance is assessed by making
predictions on the test set and calculating the Mean Squared Error (MSE)
between the predicted and actual values. This MSE is printed to provide a
measure of the model's accuracy.
___________________________________________________________________________
Algorithm 1: Model 1
1 Initialization
Import necessary libraries
2 Data Preprocessing
Define features X and target y
¿ ¿ y ← D[ area ]¿
3 Model Development
Split data into training and testing sets:
( X train , X test , y train , y test ) ← train_test_split ( X seq , y seq , test_size ¿
0.2 , random_state ¿ 42 )
Define and compile the LSTM model
model1.compile(optimizer ¿' adam ' , loss ¿' mean_squared_error ' )
4 Training
Train the model:
history1 ← model1.fit ( X train , y train , epochs ¿ 100, validation_split ¿ 0.1 )
5 Evaluation
Predict and evaluate the model:
y pred1 ← model1.predict ( X test )
y pred1 =¿ flatten ( y pred1 )
y test =¿ flatten ( y test )
MSE 1=¿ mean_squared_error ( y test , y pred1 )
___________________________________________________________________________
Data is then split into training and testing sets using train_test_split,
allocating 20% of the data for testing while ensuring reproducibility with a
fixed random state.
For the LSTM model, the architecture includes 50 units with ReLU
activation and an input shape based on the sequence length and feature
dimensions. This model also has a final dense layer with one output unit.
The model is compiled with the Adam optimizer and mean squared error
loss function, and trained for 100 epochs with a 10 % validation split. After
training, predictions are made on the test set, and the Mean Squared Error
(MSE) is calculated to assess the model's performance.
___________________________________________________________________________
Algorithm 2: Model 2
1 Data Preparation
Sequence Creation
Define a function create_sequences to generate sequences for
LSTM: create sequences ( X scaled , SEQ_LENGTH ¿=( X seq , y seq )
where
X seq [i]=¿ SEQ_LENGTH -1 ¿ }
y seq [i]= X scaled ¿ SEQ_LENGTH ¿
Sequence Length
Set the sequence length:
SEQ_LENGTH ¿ 1
Data Splitting
Split data into training and testing sets:
( X train , X test , y train , y test ) ← train_test_split ( X seq , y seq , test_size ¿
0.2 , random_state ¿ 42 )
2 Model 1: LSTM
Model Architecture:
Define the LSTM model:
model2 ¿
{
LSTM ( 50 , activation = ReLU , input_shape =( SEQ_LENGTH , X train
Dense (1)
Compile the model:
model1.compile(optimizer ¿ Adam, loss ¿ mean_squared_error)
Training
Train the model:
history 1 ← model1.fit ( X train , y train , epochs ¿ 100, validation_split ¿ 0.1 )
Evaluation
Predict and evaluate
y pred1 =¿ model1.predict ( X test )
MSE 1=¿ mean_squared_error ( y test , y predl )
3 Model 2: Feedforward Neural Network
Model Architecture
Define the Feedforward model
{
Dense ( 64 , activation = ReLU , input_shape =( X train ⋅ shape [1], ) )
model 2= Dense (32 , activation =ReLU )
Dense (1)
Compile the model
model2.compile(optimizer ¿ Adam, loss ¿ mean ¿squared_error)
Training
Train the model
history 2 ←model 2.fit ( X train ⋅reshape ( X train ⋅ shape[0], -1¿ , y train , epochs ¿
100 , validation ¿split ¿ 0.1 )
Evaluation
Predict and evaluate
y pred2 =¿ model2.predict ( X test ⋅reshape ( X test ⋅ shape [0],−1 ) )
MSE 2=¿ mean_squared ¿error ( y test , y pred2 )
___________________________________________________________________________
For training, the model is compiled with a mean squared error loss
function (`loss='mean_squared_error'`) and the configured optimizer
(`opt`). Training occurs over multiple epochs (`EPOCHS`), utilizing the
training dataset (`train_dataset`) and validating performance on the
validation dataset (`validation_dataset`). The verbosity during training
(`verbose=2`) controls the amount of output displayed.
Comparative Results
The comparative analysis of Mean Squared Error (MSE) values for the
existing method, Model 1, and Model 2 reveals notable improvements in
prediction accuracy across different environmental factors. For rainfall
prediction, both Model 1 and Model 2 outperform the existing method,
with Model 2 achieving the lowest MSE of 0.2716 compared to 0.352 for
the existing method. This indicates that the new models are better at
capturing rainfall patterns, with Model 2 showing a slight edge over Model
1.
Table 1: Comparative Mean Squared Error (MSE) Results for Rain, Wind,
and Temperature Predictions Using Existing Methods, Model 1, and Model
2
Conclusion
This study evaluates the efficacy of Long Short-Term Memory (LSTM)
networks and Feedforward Neural Networks (FNN) for time series
forecasting. The comparative analysis revealed that while both models
exhibit robust performance, there are notable differences in their
predictive accuracy and efficiency. The LSTM model demonstrated
superior performance in capturing temporal dependencies and trends
within the data, as evidenced by its lower mean squared error (MSE)
compared to the FNN. This can be attributed to LSTM's ability to retain
information over long sequences, which is particularly advantageous for
time series data. On the other hand, the FNN model, with its simpler
architecture, provided competitive results but lacked the temporal context
sensitivity that LSTM models offer.
Pseudo Code
import tensorflow as tf
from tensorflow.keras.models
import Sequential
from tensorflow.keras.layers
import Dense, InputLayer, Dropout, Conv1D, Conv2D, Flatten, Reshape,
MaxPooling1D, MaxPooling2D, AveragePooling2D, BatchNormalization,
Permute, ReLU, Softmax
from tensorflow.keras.optimizers.legacy
import Adam
EPOCHS = args.epochs or 100
LEARNING_RATE = args.learning_rate or 0.005
# If True, non-deterministic functions (e.g. shuffling batches) are not used.
# This is False by default.
ENSURE_DETERMINISM = args.ensure_determinism
# this controls the batch size, or you can manipulate the tf.data.Dataset
objects yourself
BATCH_SIZE = args.batch_size or 32
if not ENSURE_DETERMINISM:
train_dataset = train_dataset.shuffle(buffer_size=BATCH_SIZE*4)
train_dataset=train_dataset.batch(BATCH_SIZE, drop_remainder=False)
validation_dataset = validation_dataset.batch(BATCH_SIZE,
drop_remainder=False)
# model architecture
model = Sequential()
model.add(Dense(20, activation='relu'))
model.add(Dense(10, activation='relu'))
model.add(Dense(classes, name='y_pred'))
Reference
1. Butun, I., Österberg, P., & Song, H. (2019). Security of the Internet of
Things: Vulnerabilities, attacks, and countermeasures. IEEE
Communications Surveys & Tutorials, 22(1), 616-644.
2. Sonar, K., & Upadhyay, H. (2014). A survey: DDOS attack on Internet
of Things. International Journal of Engineering Research and
Development, 10(11), 58-63.
3. Dhanaraju, M., Chenniappan, P., Ramalingam, K., Pazhanivelan, S., &
Kaliaperumal, R. (2022). Smart farming: Internet of Things (IoT)-
based sustainable agriculture. Agriculture, 12(10), 1745.
4. Jayaraman, P. P., Yavari, A., Georgakopoulos, D., Morshed, A., &
Zaslavsky, A. (2016). Internet of Things platform for smart farming:
Experiences and lessons learned. Sensors, 16(11), 1884.
5. Sontowski, S., Gupta, M., Chukkapalli, S. S. L., Abdelsalam, M., Mittal, S., Joshi, A.,
& Sandhu, R. (2020, December). Cyber attacks on smart farming infrastructure. In
2020 IEEE 6th International Conference on Collaboration and Internet Computing
(CIC) (pp. 135-143). IEEE.
6. Yazdinejad, A., Zolfaghari, B., Azmoodeh, A., Dehghantanha, A.,
Karimipour, H., Fraser, E., ... & Duncan, E. (2021). A review on
security of smart farming and precision agriculture: Security
aspects, attacks, threats and countermeasures. Applied Sciences,
11(16), 7518.
7. Mohammad, Z., Qattam, T. A., & Saleh, K. (2019, April). Security
weaknesses and attacks on the internet of things applications. In
2019 IEEE Jordan International Joint Conference on Electrical
Engineering and Information Technology (JEEIT) (pp. 431-436). IEEE.
8. Sood, K., Nosouhi, M. R., Kumar, N., Gaddam, A., Feng, B., & Yu, S.
(2021). Accurate detection of IoT sensor behaviors in legitimate,
faulty and compromised scenarios. IEEE Transactions on
Dependable and Secure Computing, 20(1), 288-300.
9. Chen, Y. (2013). New approaches for calculating Moran’s index of
spatial autocorrelation. PloS one, 8(7), e68336.
10. Karim, F., Majumdar, S., Darabi, H., & Chen, S. (2017). LSTM
fully convolutional networks for time series classification. IEEE
access, 6, 1662-1669.
11. L. D. Xu, W. He, and S. Li, “Internet of Things in industries: A
survey,” IEEE Trans. Ind. Inform., vol. 10, no. 4, pp. 2233–2243, Nov.
2014.
12. Y. Liu, X. Ma, L. Shu, G. P. Hancke, and A. M. Abu-Mahfouz,
“From industry 4.0 to agriculture 4.0: Current status, enabling
technologies, and research challenges,” IEEE Trans. Ind. Inform., vol.
17, no. 6, pp. 4322–4334, Jun. 2021.
13. C. Brewster, I. Roussaki, N. Kalatzis, K. Doolin, and K. Ellis, “IoT
in agriculture: Designing a europe-wide large-scale pilot,” IEEE
Commun. Mag., vol. 55, no. 9, pp. 26–33, Sep. 2017.
14. A. Roy, P. Das, and R. Das, “Temperature and humidity
monitoring system for storage rooms of industries,” in Proc. Int.
Conf. Comput. Commun. Technol. Smart Nation, 2017, pp. 99–103.