Android Malware Detection Using

Deep Learning
Idider Meryem, Hanane Ben Daoud , Hajar Es-sabery, Imane AOUDACHT

1. Introduction:
The reliance on smartphone devices is increasing dramatically more than ever before. By
2021, the number of smart mobile devices worldwide reached 3.8 billion [1]. Moreover, more
than 72% of these devices run on the Android operating system [2]. However, Android users
seldom install antivirus software on their devices, and even those who do may not effectively
use it to detect viruses [3]. These factors make the Android system an attractive target for
cyber attackers due to the large number of users and the vast amount of valuable information
stored on these devices
Notably, as the number of users increases, so does the amount of valuable information that
cyber attackers can access. Attackers may infiltrate devices by successfully uploading malicious
applications to Google Play, which unsuspecting users then download and install, unknowingly
granting the attacker access to their data . As of the third quarter of 2020, more than 2.86
million Android applications were available for download. Additionally, an average of 482,579
malware applications were discovered each month, equating to approximately 16,000
malicious applications per day . This overwhelming number of malware applications
necessitates more sophisticated malware detection methods.
Both machine learning and deep learning techniques have proven effective in detecting
malware in Android applications and other areas of cybersecurity. In this paper, we evaluate
the performance of five classification algorithms:

• Support Vector Machine (SVM)

• K-Nearest Neighbors (KNN)
We then compare their results with a deep learning-based MLP and GRU approach using the
Android Malware dataset. The feature extraction and selection process focus on permissions,
intents, keywords and API calls, as malware applications tend to request unusual permissions
and API calls that differ from benign applications .

2.Methodology
the first phase starting with the mixture of malware and benign Android Application Package (APK)
files taken from a dataset provided for this study. Then, we extract the static features by our own
Python script written in a Jupiter Notebook environment. The features are API calls, intents, keywords
and permissions. These features are formatted and stored in a Comma-Separated Values (CSV) file as
a data frame that serves the training process
3. Preprocessing and Feature Analysis:
3.1 Dataset Overview
Our analysis covered 15,036 Android application samples with 216 features, showing an imbalanced
distribution (63% benign, 37% malware). This imbalance, while typical in security datasets, was
considered during our analysis.

3.2 Preprocessing Summary

• Converted class labels to numeric format (Benign = 0, Malware = 1)
• Handled missing values (dropped columns with >10% missing values, filled remainder with 0)
• Converted all features to numeric format for consistent analysis
• we applied Random Over-Sampling (ROS) to balance the training data, ensuring an equal
number of malware and benign samples
• features were normalized to improve model performance,
3.3 Key Feature Insights:

3.3.1 Correlation Analysis:

We identified features with significant correlation to malware classification:

Figure 1: Correlation matrix of top features with correlation > 0.5 with the target class
Figure 2: Complete feature correlation matrix showing relationships between all features

Figure 3: Extended correlation analysis of features with correlation > 0.4 with the target class

Key observations:
➢ Positively Correlated: SEND_SMS (0.55) shows the strongest positive correlation with
malware
 ➢ Negatively Correlated: Service-related APIs show strong negative correlation with malware:
 transact (-0.57)
 ServiceConnection (-0.56)
 bindService (-0.56)
 onServiceConnected (-0.56)

3.3 Density analysis:

Density analysis revealed distinct behavioral differences:

the density of the feature that have correlation Number of Features having correlation >0.4
> 0.5 with target variable

Observation:
➢ SMS & Phone Access:
 Malware shows distinct patterns in SEND_SMS usage
 READ_PHONE_STATE (0.41) has higher usage density in malware samples
➢ Service Interaction:
 Benign apps predominantly use service-related APIs (attachInterface, ServiceConnection,
bindService)
 Malware typically avoids these standard mechanisms
➢ Reflection APIs:
 Benign apps more frequently use reflection/introspection capabilities
 Malware samples show minimal usage of getMethod, cast, and getCanonicalName

3.4 Security Implications

Our analysis indicates that API usage patterns provide robust signals for malware detection:

1. Malware Behavior Profile:

a. Frequent access to phone state information
b. Distinctive SMS feature usage
c. Avoidance of standard service binding mechanisms
d. Limited use of reflection capabilities
2. Detection Strategy:
a. READ_PHONE_STATE usage combined with low service API usage strongly indicates
malicious intent
b. The bimodal distribution patterns (peaks at 0 and 1) in most features confirm binary
behavior differences between classes
4.Experimental Results:
We have conducted our experiments on two types of models, traditional machine learning classifiers
and deep learning. First, we train the classifiers using the dataset, and then we perform the testing
and evaluation. In both experiments, the classifications are based on the features extracted from
permissions,intents,keywords and API-calls.

4.1. Traditional Machine Learning:

SVM
Support Vector Machine (SVM) is a supervised learning algorithm designed for binary classification,
commonly used in fields like cybersecurity, anomaly detection, and text classification. The model
finds an optimal hyperplane that maximizes the margin between different classes, improving
generalization. It can handle both linear and non-linear classification using kernel functions. This
report provides a comprehensive review of the code implementation, training process, and
performance metrics.

1. Model Overview
1.1 SVM Models:
• Linear Kernel: SVM with a linear kernel aims to find a hyperplane that best separates the
classes.
• RBF Kernel: Uses a radial basis function to handle non-linear decision boundaries.
• Polynomial Kernel: A non-linear kernel that considers polynomial decision boundaries.

1.2 Model Evaluation Metrics:

For each model, the following performance metrics were calculated:

• Accuracy: The proportion of correct predictions.

• Precision: The proportion of true positives among all positive predictions.
• Recall: The proportion of true positives among all actual positives.
• F1 Score: The harmonic mean of precision and recall, providing a single metric for evaluation.
• ROC AUC: The area under the ROC curve, measuring the model's ability to discriminate
between classes.
• Average Precision: A summary of the Precision-Recall curve, emphasizing performance for
imbalanced classes.

The training time was also recorded to assess the computational cost of each model.
1.3 Confusion Matrix:
Confusion matrices were generated to provide a detailed view of the model’s true positives,
false positives, true negatives, and false negatives. Normalized versions were also plotted to
evaluate relative performance across different models.

2. Performance Analysis
2.1 SVM Linear Kernel

a. Classification Metrics:
The model achieved impressive results across various metrics, showcasing its effectiveness in
classification:

The model's per-class performance shows strong precision and recall for both classes:

The model achieved excellent performance across multiple metrics:

• Accuracy: 97.67% demonstrates a strong overall performance.

• Precision: 96.06% indicates the model effectively reduces false positives.
• Recall: 97.77% shows that it successfully identifies the majority of positive cases.
• F1 Score: 96.91% strikes a good balance between precision and recall.
• Prediction Time: 0.26 seconds is efficient, ensuring fast predictions.

b. Confusion Matrix:
The model's false positives (45) and false negatives (25) are relatively low, showing that it's highly
accurate in distinguishing between the two classes.
 c. ROC Curve and Precision Recall Curve:

• The ROC curve shows very good performance with an AUC of 0.995. The curve rises quickly
and stays high, meaning the model correctly identifies most positive cases without many false
positives.
• The precision-recall curve shows excellent performance with an AP (Average Precision) of
0.992. The curve maintains high precision (near 1.0) across most recall values, only dropping
at the very end when recall approaches 1.0.

2.2 SVM RBF Kernel

a. Classification Metrics:

The SVM model with RBF kernel demonstrates excellent classification performance with 98.54%
accuracy. The model shows strong precision (98.38%) and recall (97.68%), resulting in a high F1 score
of 98.03%. The confusion matrix reveals that the model made minimal errors, with only 18 false
positives and 26 false negatives out of nearly 3,000 predictions. The prediction time of 1.34 seconds
indicates good computational efficiency.

b. Confusion Matrix:

• The confusion matrix shows excellent results with 1,868 true negatives and 1,095 true
positives.
• The model made very few errors, with only 18 false positives and 26 false negatives.
• These results confirm the model's high accuracy and balanced performance across both
classes.
• The error distribution indicates the model is slightly more likely to miss positive cases
than generate false alarms.

c. ROC Curve and Precision Recall Curve:

• The SVM with RBF kernel shows outstanding performance with an AUC of 0.997. The
curve's sharp rise indicates excellent detection with few false alarms, slightly
outperforming the linear kernel model and confirming the effectiveness of the non-
linear approach.
• The precision recall curve shows superior performance with an AP of 0.997,
outperforming the linear kernel model (0.992). The precision-recall curve maintains
perfect precision across most recall values, only dropping at the very highest recall
levels. This demonstrates the RBF kernel's exceptional ability to minimize false
positives while capturing nearly all positive cases.
2.3 SVM Polynomial Kernel

a. Classification Metrics:

• SVM with Polynomial Kernel achieves 96.51% accuracy, with excellent 98.75% precision.
• The model's recall is lower at 91.79%, suggesting it misses some positive cases.
• The F1 score of 95.15% shows a good balance between precision and recall.
• Prediction time of 0.78 seconds is faster than the RBF kernel model.
• Overall, this model prioritizes precision over recall compared to the RBF kernel.

b. Confusion Matrix:
• The model correctly identified 1,873 benign cases and 1,029 malware cases.
• Only 13 benign items were incorrectly flagged as malware (false positives).
• 92 malware items were missed and classified as benign (false negatives).
• These missed malware detections explain the lower recall score and represent potential
security risks.

c. ROC Curve and Precision Recall Curve:

 • The ROC curve shows excellent performance with an AUC of 0.996. The curve rises
steeply, indicating high true positive rates with few false positives. While slightly below
the RBF kernel (0.997), it provides reliable classification with minimal trade-offs.
• The SVM with Polynomial Kernel achieves excellent performance with an AP of 0.994. The
precision remains consistently high across most recall values, only dropping at very high
recall. This confirms the model's strong precision (98.75%) while showing slightly lower
overall performance than the RBF kernel model.
• The report shows strong overall performance with 97% accuracy. For benign cases (class
0), the model has 95% precision and 99% recall. For malware (class 1), precision is
excellent at 99%, though recall is lower at 92%, confirming some malware instances are
missed. The balanced f1-scores (0.97 and 0.95) indicate good performance for both
classes despite their different distributions.

KNN
k-Nearest Neighbors (KNN) implementation for binary classification in a malware detection system.
The model classifies samples into "benign" or "malware" categories based on feature vectors using
the distance to the k-nearest data points. This report provides a comprehensive review of the code
implementation, training process, and performance metrics.
1. Technical Implementation Analysis
1.1 Model Architecture and Implementation

The implementation uses scikit-learn's KNeighborsClassifier with different hyperparameter

configurations. Key architectural components include:

• Multiple k Values: Testing various neighborhood sizes (3, 5, 7, 9, 11) to find the optimal
number of neighbors
• Uniform Weighting: Using the default uniform weighting where all points in each
neighborhood have equal influence
• Euclidean Distance: Implementing the standard Euclidean metric for measuring distances
between samples
• Brute Force Algorithm: Using scikit-learn's default algorithm for finding nearest neighbors

The implementation benefits from scikit-learn's efficient nearest neighbor finding algorithms which
handle:

• Distance calculations
• Neighbor ranking
• Voting mechanisms for classification

1.2 Training Process

The training implementation includes:

• Systematically evaluates multiple k values (3, 5, 7, 9, 11)

• Times each configuration and stores results in a structured format
• Leverages KNN's instance-based learning approach where "training" involves storing data
points
• Captures computational costs for organizing data in memory
• Enables comprehensive performance comparison across different neighborhood sizes

2. Performance Analysis
2.3 k=3

a. Classification Metrics:
The KNN model with k=3 shows excellent performance with an accuracy of 98.44%. It achieves high
precision (98.29%) and recall (97.50%), resulting in a strong F1 score of 97.90%.
 b. Confusion Matrix:

• The confusion matrix shows only 19 false positives and 28 false negatives out of 3,007 total
samples.
• Training time is extremely fast at 0.0340 seconds, which is typical for KNN which mainly
stores examples.
• Prediction time is higher at 1.0008 seconds due to the distance calculations required at
inference.

c. ROC Curve and Precision Recall Curve:

For ROC Curve:

• KNN with k=3 achieves strong performance with an AUC of 0.991

• The curve rises sharply at low false positive rates, showing excellent detection capability with
few false alarms
• Performance is slightly below the best SVM models but still demonstrates highly effective
classification

For Precision-Recall Curve:

• The model maintains near-perfect precision across most recall values (AP=0.987)
• Precision only drops at extremely high recall, indicating few false positives until attempting to
catch all positive cases
• Performance is slightly below the SVM models but still shows strong classification ability

2.3 k=5 and k = 7

a. Classification Metrics:
The KNN model with k=5 and also k= 7 shows strong performance with an accuracy of 97.87%. The
model achieves high precision (97.40%) and recall (96.88%), resulting in an F1 score of 97.14%

b. Confusion Matrix:

• 1,857 true negatives and 1,086 true positives show strong correct classifications
• Only 29 false positives and 35 false negatives demonstrate balanced error distribution
• Total error rate is very low with just 64 misclassifications out of 3,007 samples
c. ROC Curve and Precision Recall Curve:

For ROC Curve:

• KNN with k=5 &7 achieves excellent performance with an AUC of 0.993, slightly better than
k=3
• The curve shows strong detection ability with minimal false positives
• Performance approaches the best SVM models while offering faster prediction times

For Precision-Recall Curve:

• KNN achieves an excellent AP of 0.989, slightly better than k=3

• Perfect precision maintained until very high recall levels
• Sharp drop only at highest recall shows good separation between classes

2.4 k=9 and k=11

a. Classification Metrics:

Key observations:

• k=9 performs marginally better across all metrics

• Same false positives (26) in both models
• k=11 has 4 more false negatives (42 vs 38)
• Both models are very fast to train
• Prediction times are comparable
b. Confusion Matrix

• Identical performance on benign samples across both models (1860 TN, 26 FP)
• k=9 correctly identifies 4 more malware samples than k=11
• Both matrices show remarkably low false positive rates (~1.4%)
• Higher false negative rates than false positive rates in both models
• Visual intensity confirms strong class separation capabilities
• The total sample distribution shows approximately 63% benign vs 37% malware

c. ROC Curve and Precision Recall Curve:

• k=11 has higher AUC (0.996) despite lower accuracy

• Nearly identical curves despite different k-values
• Both excel at critical low FPR ranges
• 0.001 AUC difference contrasts with more visible differences in confusion matrices
• Smooth curves show stable classification behavior
Key observations from the KNN model comparison:

• k=11 slightly higher AP (0.992 vs 0.991)

• Both perfect until very high recall (>0.95)
• Identical curve shapes with steep final drops
• k=11 better AP despite lower accuracy
• Minimal difference between models
• Both achieve exceptional classification (AP >0.99)

Comparing KNN and SVM models

Model Accuracy Comparison

k Key observations :
• All models demonstrate remarkably high accuracy, with all accuracies appearing to be
above 95%
 • The differences between models are minimal, with all bars reaching nearly to the 1.0 mark
• The y-axis starts at 0, which visually emphasizes the high performance of all models
• SVM with RBF Kernel appears to have the highest accuracy (orange bar)
• KNN models show consistent performance across different k values
• The accuracy differences between k=9 and k=11 shown in your previous data (97.87% vs
97.74%) are barely perceptible on this scale
This visualization suggests that for this classification task, all the tested models perform
exceptionally well, with SVM (RBF Kernel) potentially being the top performer, though the
differences are minimal.

Model F1 Score Comparison

Key observations:
• All models demonstrate remarkably high performance, with F1 scores appearing to be above
0.95 (95%)
• The differences between models are minimal, with all bars nearly reaching the 1.0 mark
• SVM with RBF Kernel (orange bar) appears to have the highest F1 score among all models
• SVM with Polynomial Kernel (green bar) shows slightly lower performance compared to other
SVM variants
• KNN models display consistent performance across different k values (3, 5, 7, 9, and 11)
• The performance differences between KNN models with varying k values are barely
perceptible at this scale
 • The y-axis starts at 0, which provides proper context but also visually emphasizes the high
performance of all models

This visualization suggests that for this particular classification task, all tested models perform
exceptionally well, with SVM (RBF Kernel) potentially being the optimal choice, though the
performance differences are marginal and might not be statistically significant.

Model ROC AUC Comparison

Key observations:

• All models demonstrate excellent discriminative ability with ROC AUC values approaching 1.0
(perfect classification)
• The performance differences between all models are extremely minimal
• SVM with RBF Kernel (orange bar) appears to have a marginally higher ROC AUC score
• SVM with Linear Kernel and Polynomial Kernel show comparable performance to the RBF
variant
• KNN models maintain consistent performance regardless of k value (3, 5, 7, 9, or 11)
• The y-axis begins at 0, providing appropriate context while highlighting the strong
performance across all models

This visualization reveals that for this classification task, all tested models perform exceptionally well
in terms of ROC AUC, suggesting they all effectively distinguish between classes. The nearly identical
performance across models indicates that model selection could reasonably be based on other
factors such as computational efficiency, interpretability, or specific use case requirements rather
than ROC AUC performance alone.

Model Training Time Comparison

Key observations:

• The SVM models show significant differences in training time, with a clear progression from
fastest to slowest

• SVM with Linear Kernel is the fastest SVM model (~25 time units)

• SVM with RBF Kernel has a moderate training time (~34 time units)

• SVM with Polynomial Kernel requires substantially more time to train (~50 time units)

• The KNN models appear to have negligible training times, with bars so small they're barely
visible on the chart

• The training time difference between SVM models is substantial - the Polynomial Kernel takes
about twice as long as the Linear Kernel

This visualization highlights an important trade-off: while the performance metrics (F1 score and ROC
AUC) showed minimal differences between models, the computational efficiency varies dramatically.
KNN models clearly have a significant advantage in terms of training speed compared to all SVM
variants. Among the SVM models, the Linear Kernel offers the best computational efficiency, while
the Polynomial Kernel demands considerably more computational resources despite showing similar
performance metrics.
Key observations:

• All three models show nearly identical, excellent performance across all metrics

• Performance values consistently approach 1.0 (perfect) for Precision, Accuracy, ROC AUC, F1
Score, and Recall

• The lines completely overlap, making it difficult to distinguish between models

• All models display balanced performance with no weak areas

• High precision and recall values indicate minimal false positives and false negatives

This visualization confirms that all three models perform exceptionally well with negligible
differences, suggesting model selection should be based on other factors like training speed or
interpretability rather than performance metrics.

KNN: Impact of k on Performance Metrics

Key observations:

• Accuracy (blue): Highest at k=3 (~0.984), then drops to ~0.979 for k=5-9, and slightly
decreases again at k=11 (~0.978)
• Precision (green): Peaks at k=3 (~0.983), drops significantly at k=5-7 (~0.974), then partially
recovers for k=9-11 (~0.976)
• Recall (red): Shows a clear downward trend as k increases, starting highest at k=3 (~0.975)
and ending lowest at k=11 (~0.963)
• F1 Score (purple): Follows a similar pattern to accuracy and recall, with highest value at k=3
(~0.979), dropping at k=5 (~0.971), and lowest at k=11 (~0.969)
• ROC AUC (yellow): Shows the opposite trend, starting lowest at k=3 (~0.991) and generally
increasing to its highest at k=11 (~0.996)
The plots reveal that k=3 provides optimal performance for accuracy, precision, recall, and F1 score,
while k=11 maximizes ROC AUC. This sugge

4.2 Deep learning:

GRU
Gated Recurrent Unit (GRU) neural network designed for binary classification in what appears to be a
malware detection system. The model classifies samples into "benign" or "malware" categories using
feature vectors derived from the dataset. This report provides a comprehensive review of the code
implementation, training process, performance metrics.
1. Technical Implementation Analysis
1.1 Data Preparation

The implementation begins with data preparation steps that follow standard machine learning
practices: This approach follows a proper pipeline:

• Converting pandas DataFrames to NumPy arrays

• Converting NumPy arrays to PyTorch tensors with appropriate types (FloatTensor for features,
LongTensor for labels)
The mini-batch training paradigm is implemented via PyTorch's DataLoader

The batch size of 64 is a reasonable choice that balances computational efficiency with gradient
stability. Enabling shuffle ensures randomized training examples, which helps prevent the model from
developing biases based on the order of training data.

1.2 Model Architecture

Notable architectural decisions include:

• Reshape Function: The model treats each input vector as a sequence of length 1, allowing
the GRU to process it even though the data may not be inherently sequential.
• Single GRU Layer: With batch_first=True for intuitive batch dimension handling.
• Hidden Size: Set to 32, which is sufficient for capturing feature relationships without
excessive complexity.
• Output Layer: A fully connected layer maps the GRU's output to two classes.
• Additional predict_proba Method: Provides probability outputs, which is valuable for
threshold adjustments and confidence assessment.

1.3 Training Process

The training implementation follows the standard PyTorch paradigm

The training process includes:

• Loss Function: Cross-entropy loss, appropriate for classification tasks

• Optimizer: Adam with a learning rate of 0.001, which is a suitable default choice
• Epoch Tracking: Loss values are logged per epoch, showing clear convergence
• Gradient Zeroing: Proper handling of gradients between batches
• Time Measurement: Training duration is measured and reported
2. Performance Analysis
2.1 Training Progression

The training logs show consistent and significant reduction in loss across epochs:

The most significant reduction occurs between epochs 1 and 2, after which the loss continues to
decrease steadily but at a slower rate. This pattern suggests:

• The model learns most of the pattern recognition in the first few epochs
• There is no apparent overfitting within these 10 epochs as the loss continues to decrease
• Training for 10 epochs appears sufficient for this task

The total training time of 6.92 seconds indicates efficient implementation and reasonably sized
dataset.

2.2 Performance Metrics

The model achieved excellent performance across multiple metrics:

The model's per-class performance shows balanced classification capabilities:

Key observations:

• The difference between training (99.18%) and test (98.20%) accuracy is minimal, suggesting
good generalization
• Class balance in the test set: 1886 benign samples (63%) vs. 1121 malware samples (37%)
• The model performs slightly better on benign samples (class 0) with 99% precision compared
to 97% for malware
• High recall for malware samples (98%) indicates the model rarely misses actual threats
• The overall F1 score of 97.60% demonstrates balanced performance between precision and
recall

2.3 ROC and Precision-Recall Curves Analysis

The performance of the GRU model is further validated through ROC and Precision-Recall curves,
which provide deeper insights into the classification performance across different thresholds.

2.3.1 ROC Curve Analysis

The ROC (Receiver Operating Characteristic) curve plots the True Positive Rate (sensitivity) against the
False Positive Rate (1-specificity) at various threshold settings:
Key observations from the ROC curve:

• Near-Perfect Performance: The curve hugs the top-left corner of the plot, indicating
exceptional discriminative ability
• AUC (Area Under Curve) = 1.00: This perfect score confirms the model's outstanding ability
to distinguish between malware and benign classes
• Steep Initial Rise: The curve shows that the model achieves a very high true positive rate
even at extremely low false positive rates
• Threshold Flexibility: The curve shape suggests the model maintains excellent performance
across a wide range of classification thresholds

This ROC curve performance indicates that the model can be tuned to prioritize either high detection
rates or low false alarm rates with minimal performance trade-offs.

2.3.2 Precision-Recall Curve Analysis

The Precision-Recall curve illustrates the trade-off between precision (positive predictive value) and
recall (sensitivity) at different threshold settings:
Key observations from the Precision-Recall curve:

• High Performance Region: The curve maintains near-perfect precision (close to 1.0) across a
wide range of recall values (up to approximately 0.9)
• Sharp Drop: There's a precipitous drop in precision only at very high recall values (above 0.9)
• AUC (Area Under Curve) = 1.00: The perfect area under the PR curve confirms the model's
robust performance
• Operational Threshold Selection: The curve suggests that a threshold can be selected to
achieve recall up to about 0.9 while maintaining near-perfect precision

The PR curve is particularly important for malware detection where class imbalance often exists. The
high area under the curve indicates that the model maintains high precision even when tuned for
higher recall, which is crucial for security applications where false negatives (missed malware) can be
costly.

3. Discussion
3.1 Strengths of the Implementation

1. Effective Architecture Choice: Using GRU for this classification task proves highly effective,
even though the data may not be inherently sequential. This suggests that the GRU's gating
mechanisms help in focusing on important features.
2. Computational Efficiency: The model trains quickly (under 7 seconds) while achieving high
accuracy, making it practical for deployment in real-world scenarios.
3. Balanced Performance: The model maintains similarly high performance for both classes,
which is crucial for malware detection where both false positives and false negatives can be
problematic.
 4. Proper Engineering Practices: The code follows good software engineering principles,
including:
a. Clear separation of model definition, training, and evaluation
b. Proper tensor type handling
c. Use of torch.no_grad() for inference
d. Batch processing for efficiency

3.2 Potential Improvements

1. Validation Set: The current implementation doesn't include a validation set during training,
which could help monitor for overfitting and inform early stopping decisions.
2. Hyperparameter Tuning: Several model parameters could be optimized:
a. Hidden layer size (currently 32)
b. Number of GRU layers (currently 1)
c. Learning rate (currently 0.001)
d. Batch size (currently 64)
3. Threshold Selection Strategy: Given the curves' characteristics, developing an explicit
threshold selection strategy could optimize the model for specific operational requirements
(e.g., maximizing recall while maintaining a minimum precision).
4. Regularization Techniques: While overfitting doesn't appear to be an issue, adding dropout
or weight regularization could potentially further improve generalization.
5. Model Persistence: Adding functionality to save and load the trained model would be
beneficial for deployment scenarios.
6. Feature Analysis: Understanding which features contribute most to the classification decision
could provide valuable insights for security applications.

4. Conclusion
The implemented GRU model demonstrates exceptional performance in classifying samples as benign
or malware, achieving over 98% accuracy on the test set with balanced precision and recall. Both the
ROC and Precision-Recall curves confirm this excellent performance, with AUC values of 1.00,
indicating perfect or near-perfect classification capabilities.

The architecture efficiently learns patterns in the feature space, with minimal difference between
training and test performance indicating good generalization capabilities. The model's high recall rate
for malware detection (98.04%) is particularly valuable in security contexts where missing a threat
can have serious consequences.

The ROC curve analysis reveals that the model achieves high true positive rates even at extremely low
false positive thresholds, while the PR curve demonstrates that precision remains high across a wide
range of recall values. These characteristics make the model highly adaptable to different operational
requirements in security contexts.

This implementation provides a strong foundation for malware detection systems, with potential for
further enhancement through regularization techniques, hyperparameter tuning, and additional
feature engineering. The balanced performance across classes and metrics suggests the model would
be reliable in production environments where both false positives and false negatives need to be
minimized.
The success of this GRU-based approach also indicates that recurrent neural network architectures
can effectively capture complex patterns in security-related data, even when not working with
traditionally sequential data. This suggests that similar approaches could be applied to other security
classification tasks beyond malware detection.

MLP
Multi-Layer Perceptron (MLP) implementation for binary classification in a malware detection system.
The model classifies samples into "benign" or "malware" categories using feature vectors derived
from the dataset. This report provides a comprehensive review of the code implementation, training
process, performance metrics.

1. Technical Implementation Analysis

1.1 Model Architecture and Implementation

The implementation uses scikit-learn's MLPClassifier with a simple architecture.

Key architectural components include:

• Single Hidden Layer: With 10 neurons, providing sufficient complexity while maintaining
computational efficiency
• Max Iterations: Set to 1000, ensuring adequate training time for convergence
• Random State: Fixed at 42 for reproducibility of results

The model uses scikit-learn's default activation function (ReLU) and solver (adam), which are well-
suited for this classification task. The implementation benefits from scikit-learn's robust handling of:

• Feature scaling
• Weight initialization
• Adaptive learning rate
• Mini-batch training

1.2 Training Process

The training implementation is concise and follows scikit-learn conventions

This simple approach leverages scikit-learn's optimized implementation, which handles:

• Mini-batch gradient descent

• Learning rate scheduling
• Early stopping if convergence is reached
• Progress monitoring during training

While the code doesn't show the specific training progression (loss values per epoch), scikit-learn's
MLPClassifier includes internal monitoring for convergence.
2. Performance Analysis
2.1 Classification Metrics

The model achieved excellent performance across multiple metrics:

The model's per-class performance shows balanced classification capabilities:

Key observations:

• The difference between training (99.90%) and test (98.30%) accuracy is minimal, suggesting
good generalization
• Class balance in the test set: 1886 benign samples (63%) vs. 1121 malware samples (37%)
• The model performs consistently well on both benign samples (class 0) and malware samples
(class 1)
• High recall for malware samples (97.15%) indicates the model rarely misses actual threats
• The overall F1 score of 97.71% demonstrates balanced performance between precision and
recall

3.2 Confusion Matrix Analysis

The confusion matrix provides a detailed view of the model's classification performance:
From the confusion matrix we can observe:

• True Negatives (Benign correctly classified): 1867 (99% of actual benign samples)
• False Positives (Benign misclassified as malware): 19 (1% of actual benign samples)
• False Negatives (Malware misclassified as benign): 32 (3% of actual malware samples)
• True Positives (Malware correctly classified): 1089 (97% of actual malware samples)

This matrix confirms the model's strong performance, with very few misclassifications in either class.
The slightly higher rate of false negatives (3%) compared to false positives (1%) suggests the model is
marginally more conservative in flagging samples as malware.

2.3 ROC and Precision-Recall Curves Analysis

The performance of the MLP model is further validated through ROC and Precision-Recall curves,
which provide deeper insights into the classification performance across different thresholds.

2.3.1 ROC Curve Analysis

The ROC (Receiver Operating Characteristic) curve plots the True Positive Rate (sensitivity) against the
False Positive Rate (1-specificity) at various threshold settings:
Key observations from the ROC curve:

• Near-Perfect Performance: The curve hugs the top-left corner of the plot, indicating
exceptional discriminative ability
• AUC (Area Under Curve) = 1.00: This perfect score confirms the model's outstanding ability
to distinguish between malware and benign classes
• Steep Initial Rise: The curve shows that the model achieves a very high true positive rate
even at extremely low false positive rates
• Threshold Flexibility: The curve shape suggests the model maintains excellent performance
across a wide range of classification thresholds

This ROC curve performance indicates that the model can be tuned to prioritize either high detection
rates or low false alarm rates with minimal performance trade-offs.

2.3.2 Precision-Recall Curve Analysis

The Precision-Recall curve illustrates the trade-off between precision (positive predictive value) and
recall (sensitivity) at different threshold settings:
Key observations from the Precision-Recall curve:

• High Performance Region: The curve maintains near-perfect precision (close to 1.0) across a
wide range of recall values (up to approximately 0.95)
• Sharp Drop: There's a precipitous drop in precision only at very high recall values (above
0.95)
• AUC (Area Under Curve) = 1.00: The perfect area under the PR curve confirms the model's
robust performance
• Operational Threshold Selection: The curve suggests that a threshold can be selected to
achieve recall up to about 0.95 while maintaining near-perfect precision

The PR curve is particularly important for malware detection where class imbalance often exists. The
high area under the curve indicates that the model maintains high precision even when tuned for
higher recall, which is crucial for security applications where false negatives (missed malware) can be
costly.

3. Discussion
3.1 Strengths of the Implementation

1. Simplicity and Effectiveness: Using scikit-learn's MLPClassifier with just 10 neurons in a single
hidden layer achieves excellent performance, showing that a relatively simple architecture is
sufficient for this classification task.
2. Excellent Generalization: The minimal gap between training (99.90%) and test (98.30%)
accuracy indicates that the model generalizes well to unseen data.
3. Balanced Performance Across Classes: The model maintains similarly high precision and
recall for both benign and malware classes, crucial for a security application where both false
positives and false negatives carry costs.
 4. Exceptional ROC and PR Curve Performance: The perfect or near-perfect AUC scores for both
curves demonstrate the model's robustness across different operating thresholds.
5. Implementation Efficiency: The scikit-learn implementation provides a clean, efficient
solution with built-in handling of important aspects like mini-batch training and adaptive
learning rates.

3.2 Potential Improvements

1. Architecture Exploration: Experimenting with different hidden layer sizes and multiple
hidden layers might yield even better performance or improved generalization.
2. Hyperparameter Tuning: Systematic tuning of hyperparameters could further optimize
performance:
a. Learning rate
b. Regularization parameters (alpha)
c. Batch size
d. Activation functions
3. Threshold Selection Strategy: Given the ROC and PR curves' characteristics, developing an
explicit threshold selection strategy could optimize the model for specific operational
requirements.
4. Feature Selection/Engineering: Analyzing feature importance and possibly reducing
dimensionality might improve both performance and training speed.
5. Ensemble Methods: Combining the MLP with other classifiers in an ensemble might improve
robustness and performance.
6. Explainability Methods: Implementing techniques to explain the model's decisions would be
valuable in a security context where understanding why a sample was flagged as malware is
important.

4. Comparison with GRU Model

The MLP model shows comparable performance to the GRU model examined previously:

Metric MLP GRU

Test Accuracy 98.30% 98.20%
Precision 98.29% 97.17%
Recall 97.15% 98.04%
F1 Score 97.71% 97.60%

Key observations:

• The MLP achieves slightly higher overall accuracy and precision

• The GRU model has slightly better recall for malware detection
• Both models achieve exceptional ROC AUC and PR AUC scores (1.00)
• The MLP model has a simpler architecture (10 neurons in one hidden layer vs. GRU's
recurrent design)

This comparison suggests that for this particular malware detection task, a simple feedforward neural
network is sufficient to achieve state-of-the-art performance, and the additional complexity of a
recurrent architecture like GRU offers minimal advantage.
5. Conclusion
The implemented MLP model demonstrates exceptional performance in classifying samples as benign
or malware, achieving 98.30% accuracy on the test set with well-balanced precision and recall
metrics. Both the ROC and Precision-Recall curves confirm this excellent performance, with AUC
values of 1.00, indicating perfect or near-perfect classification capabilities.

The architecture efficiency is particularly noteworthy - with just 10 neurons in a single hidden layer,
the model achieves performance comparable to or slightly better than more complex architectures
like GRU. This suggests that the feature representation of the malware samples is already highly
discriminative, allowing even relatively simple models to perform excellently.

The confusion matrix shows that the model makes very few errors in either direction, with a slightly
higher tendency toward false negatives (missing 3% of malware) than false positives (incorrectly
flagging 1% of benign samples as malware). This slight bias might be appropriate for many
operational contexts where excessive false alarms can lead to "alert fatigue."

The ROC and PR curves demonstrate that the model maintains excellent discriminative power across
different threshold settings, allowing operators to adjust the false positive/negative trade-off
according to specific security requirements without significant performance degradation.

This implementation provides a strong foundation for malware detection systems, with the added
benefit of using widely available, well-optimized libraries (scikit-learn) that facilitate easy deployment
and maintenance. The simplicity and effectiveness of the MLP approach make it an excellent
candidate for production environments where both performance and resource efficiency are valued.

4.3 Test: