WEATHER PATTERNS

ANALYSIS AND
PREDICTION
PRESENTED BY SANDESH SATISH DUBE
SUBMITTED ON 10/12/2024
 INTRODUCTION
❑ AIM OF WEATHER PATTERNS ANALYSIS AND PREDICTION:
➢ IMPROVING FORECAST ACCURACY: THE PRIMARY AIM IS TO PROVIDE ACCURATE AND TIMELY WEATHER FORECASTS TO HELP
PEOPLE PREPARE FOR CHANGING CONDITIONS. ACCURATE FORECASTS HELP IN VARIOUS SECTORS LIKE AGRICULTURE,
TRANSPORTATION, EMERGENCY RESPONSE, AND DAILY ACTIVITIES.

➢ UNDERSTANDING ATMOSPHERIC PROCESSES: BY STUDYING WEATHER PATTERNS, METEOROLOGISTS AIM TO BETTER

UNDERSTAND HOW DIFFERENT ATMOSPHERIC FORCES, SUCH AS PRESSURE SYSTEMS, JET STREAMS, AND OCEAN CURRENTS,
INTERACT TO CREATE VARIOUS WEATHER PHENOMENA LIKE STORMS, HEAT WAVES, AND RAINFALL.

❑DATASET ANALYSIS OVERVIEW:

➢ SUMMARY STATISTICS:
• DEW POINT (°C): RANGES FROM 1°C TO 28°C (MEAN: 16.6°C).
• HUMIDITY (%): RANGES FROM 6% TO 100% (MEAN: 36.3%).
• PRESSURE (HAP): RANGES FROM 994 TO 1026 HPA (MEAN: 1007.7 HPA).
• TEMPERATURE (°C): RANGES FROM 12°C TO 45°C (MEAN: 30.8°C).
• VISIBILITY (KM): RANGES FROM 0.2 KM TO 55 KM (MEAN: 2.4 KM).
 EXPLORATORY DATA ANALYSIS (EDA)
NUMERICAL DATA: Key Observations from Exploratory Data Analysis (EDA)
• TEMPERATURE (°C): General Statistics:
The dataset spans 730 days, with unique daily observations.
MAXIMUM: 45°C , MINIMUM: 12°C
MEAN: 30.8°C MEDIAN: 32°C Weather conditions are categorized into 15 distinct types, with "Haze" being
the most frequent.
• HUMIDITY (%): Temperature ranges from 12°C to 45°C, while visibility varies significantly,
MAXIMUM: 100% , MINIMUM: 6% , ranging from 0.2 km to 55 km.
MEAN: 36.3% Rain Presence: Rain is a rare event, with only about 2.47% of the days
experiencing rain.
• PRESSURE (HPA): Data Completeness: There are no missing values in the dataset.
Distribution Insights: The most common wind direction is "WNW.“
RANGE: 994 TO 1026 HPA , MEAN: 1007.7 HPA
Average humidity is about 36%, while pressure averages around 1007 hPa.
• VISIBILITY (KM):.MAXIMUM: 55 K. , MINIMUM: 0.2 KM ,
MEAN: 2.4 KM

• CATEGORICAL VARIABLES:
• WEATHER CONDITION: 15 UNIQUE WEATHER CONDITIONS OBSERVED .
• WIND DIRECTION (COMPASS): 16 UNIQUE WIND DIRECTIONS RECORDED.

➢ RAIN PRESENCE:
• NO RAIN (0): 712 OCCURRENCES.
• RAIN (1): 18 OCCURRENCES.
 METHODOLOGY
➢ K-NN CLASSIFICATION
OBJECTIVE : PREDICT WHETHER IT WILL RAIN BASED ON WEATHER FEATURES.
STEPS.

DATA PREPROCESSING:
• STANDARDIZE NUMERICAL FEATURES (DEW POINT, HUMIDITY, PRESSURE, ETC.) TO ENSURE EQUAL WEIGHT.
• ENCODE CATEGORICAL DATA (LIKE WIND DIRECTION) USING TECHNIQUES SUCH AS ONE-HOT ENCODING.

DISTANCE CALCULATION:
• COMPUTE THE DISTANCE BETWEEN DATA POINTS (E.G., EUCLIDEAN DISTANCE).

SELECTING NEAREST NEIGHBORS:

• FOR A GIVEN TEST INSTANCE, IDENTIFY THE KK NEAREST NEIGHBORS BASED ON THE CALCULATED DISTANCES.

PREDICTION:
• USE A MAJORITY VOTING MECHANISM AMONG THE KK NEIGHBORS TO CLASSIFY RAIN PRESENCE (0 OR 1).

➢ K-MEANS CLUSTERING
OBJECTIVE: GROUP WEATHER PATTERNS INTO CLUSTERS TO IDENTIFY COMMON PATTERNS .

STEPS:
• INITIALIZATION: RANDOMLY SELECT KK INITIAL CLUSTER CENTROIDS FROM THE DATASET.
• ASSIGNMENT: ASSIGN EACH DATA POINT TO THE NEAREST CENTROID BASED ON A DISTANCE METRIC.
• RECOMPUTATION: RECALCULATE THE CENTROIDS AS THE MEAN OF ALL POINTS IN EACH CLUSTER.
• REPEAT: ITERATE THE ASSIGNMENT AND RECOMPUTATION STEPS UNTIL CLUSTER ASSIGNMENTS STABILIZE OR A THRESHOLD IS MET.
 RESULTS
• K-NN CLASSIFICATION RESULTS: Predicte
Predicte
d No
d Rain
CLASSIFICATION REPORT Rain
NO RAIN (0):
• PRECISION: 0.99, RECALL: 1.00, F1-SCORE: 0.99 Actual
215 1
RAIN (1): No Rain
• PRECISION: 0.00, RECALL: 0.00, F1-SCORE: 0.00
ACCURACY Actual
• OVERALL ACCURACY: 98 % 3 0
Rain

• K MEANS CLUSTERING RESULTS:

Cluster Dew Point Humidity Pressure Temperature Visibility

(°C) (%) (hPa) (°C) (km)
Cluster 0 20.70 36.22 1002.08 35.92 2.69
Cluster 1 12.14 36.51 1014.02 25.09 1.96
Cluster 2 20.00 25.00 1005.00 38.00 55.00
 INSIGHTS
➢ INSIGHTS
❑ THE MODEL IS HIGHLY SKEWED TOWARDS PREDICTING Contribution of Techniques:
"NO RAIN" DUE TO THE DATASET'S EDA: Helped identify trends (e.g., rare rain events,
IMBALANCE. TECHNIQUES LIKE OVERSAMPLING OR UNDER-SAMPLING
common haze conditions).
MAY BE NEEDED TO IMPROVE PERFORMANCE ON THE MINORITY CLASS.
K-NN Classification: Showed that predicting rare
• PERFORMANCE METRICS: events like rain requires handling class imbalance.
▪ OVERALL ACCURACY IS HIGH AT 99%, BUT THIS IS HEAVILY K-Means Clustering : Revealed distinct weather
INFLUENCED BY THE CLASS IMBALANCE IN THE DATASET (RAIN IS RARE). patterns and potential outliers, which may help in
▪ THE MODEL HAS DIFFICULTY PREDICTING RAIN PRESENCE better understanding regional climate characteristics .
(CLASS 1), WITH PRECISION, RECALL, AND F1-SCORES AT 0 FOR THIS
CLASS.
• CONFUSION MATRIX:

• TRUE NEGATIVES: 144 (CORRECTLY PREDICTED NO RAIN).

• FALSE NEGATIVES: 2 (ACTUAL RAIN MISCLASSIFIED AS NO RAIN).
• INSIGHTS:
• K-MEANS EFFECTIVELY IDENTIFIES DIFFERENT WEATHER PATTERNS BASED ON
FEATURES LIKE TEMPERATURE AND HUMIDITY.

• CLUSTER 2 HIGHLIGHTS A POTENTIAL OUTLIER OR RARE WEATHER CONDITIONS

WITH UNUSUALLY HIGH VISIBILITY.
 CHALLENGES AND RECOMENDATION
• CHALLENGES: RECOMMENDATIONS:
CLASS IMBALANCE IN RAIN PREDICTION:
ADDRESS CLASS IMBALANCE:
THE RAIN PRESENCE DATA WAS HIGHLY IMBALANCED, WITH ONLY 2.47% Use techniques like Synthetic Minority Oversampling Technique (SMOTE), class-weight
OF DAYS HAVING RAIN. THIS LED TO POOR MODEL PERFORMANCE IN adjustments, or undersampling to balance the dataset for better rain prediction performance.
DATA QUALITY IMPROVEMENT:
PREDICTING RAIN DESPITE HIGH OVERALL ACCURACY.
Investigate outliers like unusually high visibility values to determine if they reflect real conditions
OUTLIERS IN VISIBILITY: or data errors.Collect more data points for rare events like rain to improve model robustness.
EXTREMELY HIGH VISIBILITY VALUES IN ONE CLUSTER (E.G., 55 KM) SUGGEST FEATURE ENGINEERING:
Introduce additional weather-related features, such as wind speed or historical averages, to
POTENTIAL DATA ANOMALIES OR UNIQUE WEATHER CONDITIONS, WHICH
enrich the dataset. Explore feature selection methods to identify the most impactful predictors.
COULD SKEW CLUSTER INTERPRETATIONS. MODEL SELECTION:
FEATURE SCALING: Experiment with advanced classification models, such as Random Forests or Gradient Boosting,
DIFFERENT FEATURE RANGES REQUIRED CAREFUL STANDARDIZATION TO ENSURE
which can handle imbalanced datasets better. For clustering, consider using hierarchical
clustering
MODELS LIKE K-NN AND K-MEANS PERFORMED CORRECTLY. or Gaussian Mixture Models for more nuanced groupings.
INTERPRETABILITY OF CLUSTERS: CROSS-VALIDATION:
K-MEANS CLUSTERING RESULTS ARE SENSITIVE TO THE CHOICE OF FEATURES AND
Use cross-validation for a more reliable estimate of model performance, especially given the
imbalanced nature of the dataset.
THE NUMBER OF CLUSTERS. DETERMINING THE OPTIMAL NUMBER OF CLUSTERS
SCALABILITY AND AUTOMATION:
(E.G., USING THE ELBOW METHOD) ADDS COMPUTATIONAL OVERHEAD. Automate preprocessing and analysis steps for scalability in future projects.
Leverage cloud computing for handling larger datasets or computationally intensive tasks.
 Conclusion
PURPOSE AND MAIN FINDINGS: THIS PROJECT AIMED TO ANALYZE WEATHER DATA TO IDENTIFY TRENDS, PREDICT RAIN PRESENCE, AND UNCOVER PATTERNS USING MACHINE
LEARNING TECHNIQUES. THE KEY FINDINGS INCLUDE:

RAIN IS A RARE EVENT, OCCURRING ON ONLY 2.47% OF THE DAYS.

PREDICTIVE MODELING USING K-NN FACED CHALLENGES DUE TO CLASS IMBALANCE, RESULTING IN POOR PERFORMANCE IN DETECTING RAIN.

K-MEANS CLUSTERING IDENTIFIED DISTINCT WEATHER PATTERNS, REVEALING GROUPS WITH VARYING TEMPERATURE, HUMIDITY, AND VISIBILITY CHARACTERISTICS.

EDA HIGHLIGHTED KEY FEATURES LIKE THE DOMINANCE OF HAZE, MODERATE HUMIDITY LEVELS, AND A WIDE RANGE OF VISIBILITY AND TEMPERATURES.

BROADER IMPLICATIONS:
CLIMATE INSIGHTS: THE CLUSTERING RESULTS PROVIDE INSIGHTS INTO REGIONAL WEATHER DYNAMICS, WHICH COULD AID IN CLIMATE ANALYSIS, AGRICULTURAL PLANNING, AND
DISASTER PREPAREDNESS.

RAIN PREDICTION: IMPROVING RAIN PREDICTION MODELS COULD BENEFIT INDUSTRIES LIKE AGRICULTURE AND TRANSPORTATION, WHERE WEATHER PLAYS A CRITICAL ROLE.

DATA QUALITY: THE PRESENCE OF ANOMALIES (E.G., EXTREMELY HIGH VISIBILITY VALUES) UNDERSCORES THE IMPORTANCE OF HIGH-QUALITY, CONSISTENT DATA FOR RELIABLE INSIGHTS.

VALUE OF DATA SCIENCE AND AI:

ENHANCED DECISION-MAKING: MACHINE LEARNING TECHNIQUES LIKE K-NN AND K-MEANS PROVIDE ACTIONABLE INSIGHTS THAT CAN GUIDE POLICY, RESOURCE ALLOCATION, AND
OPERATIONAL STRATEGIES.

SCALABILITY: AI-DRIVEN APPROACHES ALLOW FOR ANALYZING LARGE-SCALE, MULTI-DIMENSIONAL DATASETS EFFICIENTLY, UNCOVERING PATTERNS THAT MIGHT BE MISSED BY MANUAL
METHODS.

VERSATILITY: THE SAME TECHNIQUES CAN BE APPLIED ACROSS DOMAINS, FROM WEATHER PREDICTION TO HEALTHCARE, SHOWCASING THE TRANSFORMATIVE POWER OF DATA SCIENCE.

BY COMBINING DOMAIN KNOWLEDGE WITH DATA SCIENCE TOOLS, THIS PROJECT DEMONSTRATED THE POTENTIAL OF AI IN SOLVING REAL-WORLD CHALLENGES, FOSTERING DATA-
DRIVEN INNOVATION ACROSS SECTORS.
REFERENCES
TOOLS AND SOFTWARE:
• ORANGE DATA MINING: USED FOR VISUAL ANALYSIS AND IMPLEMENTING MACHINE LEARNING
TECHNIQUES LIKE CLASSIFICATION AND CLUSTERING.
• MICROSOFT EXCEL: UTILIZED FOR INITIAL DATA INSPECTION, CLEANING, AND EXPLORATORY DATA
ANALYSIS.
WEBSITES AND ARTICLES:
• ORANGE DATA MINING DOCUMENTATION:.
REFERENCED FOR UNDERSTANDING WORKFLOWS AND VISUAL TOOLS FOR CLASSIFICATION AND
CLUSTERING.
• WEATHER DATA ANALYSIS ON KAGGLE:
CONSULTED FOR INSIGHTS ON WORKING WITH WEATHER DATASETS AND APPLYING MACHINE
LEARNING TECHNIQUES.