Presentation on

Big Data Applications using ML

Submitted by-
1. Amit Acharjee 202010007004
2.Anandam Paul 202010007005
3.Sourajit Deb 202010007043
4.Tusharadri Paul 202010007049
5.Kukipriya Kutum 202010007028

Department of Computer Science and Engineering

Barak Valley Engineering College, Karimganj
Contents
 Introduction
 Importance of ML in Big Data
 Applications of Big Data using ML
 Case Studies
 Challenges and Consideration
 Future Trends
 Conclusion
 References
 3
Introduction
 Big Data - Big Data refers to extremely large
and complex datasets that require specialized
tools and techniques for storage, processing and
analysis.

 Machine Learning – It is a branch of artificial

intelligence that enables computers to learn from
data and improve their performance on tasks
without being explicitly programmed

Figure 1:
Relation of big data
 4
Importance Of Machine Learning In Big Data
 Pattern recognition: Machine learning algorithms can identify patterns, trends, and
correlations within vast amounts of data that would be difficult or impossible for humans to
recognize manually. Eg: SVM.

 Predictive analytics: Machine learning models can analyze historical data to predict future
outcomes, helping businesses anticipate trends, customer behavior, and market changes. Eg:
gradient boosting machines (GBM).

 Anomaly detection: Machine learning techniques can automatically detect unusual patterns
or anomalies in big data, such as fraudulent transactions, network intrusions, or equipment
failures. Eg: isolation forest.
IMPORTANCE OF MACHINE 5

LEARNING IN BIG DATA (contd..)

 Optimization: Machine learning algorithms can optimize processes and systems
by analyzing large volumes of data to identify inefficiencies, streamline
workflows, and improve resource allocation. Eg: Root Mean Square Propagation.

 Automation: Machine learning enables automation of repetitive tasks such as data

cleaning, feature engineering, and model training. Eg: AutoML.

 Real-time Decision Making: Machine learning algorithms can analyze streaming

data in real-time and make instant decisions or recommendations. Eg: Incremental
Learning using decision tree.
 6
Applications
Big Data applications using ML are diverse
and impactful across various industries.
Some of them are mentioned below-
 Healthcare
 Recommendation System
 Human Resource and talent
management
 Supply Chain optimization
 Finance
 E-commerce
Figure 2:
Big data
applications
 7
Case Study: Recommendation System
1.Collaborative Filtering:
1. Collaborative filtering recommends items to users based on their similarity to other users or items.
There are two main approaches:
1. User-based collaborative filtering: Recommends items to a user that similar users have liked.
2. Item-based collaborative filtering: Recommends items that are similar to the ones the user has liked
in the past.
2. Techniques: Singular Value Decomposition (SVD), Alternating Least Squares (ALS), and Matrix
Factorization.
2.Content-Based Filtering:
1. Content-based filtering recommends items to users based on the characteristics or features of the items
and the user's preferences.
2. Techniques: Vectorization of item features (e.g., TF-IDF for text data, image embeddings for image
data) and similarity measures.
3.Hybrid Recommender Systems:
1. Hybrid recommender systems combine collaborative filtering and content-based filtering approaches to
leverage their respective strengths and provide more accurate recommendations.
2. Techniques: Weighted combination of recommendation scores from different algorithms, cascade
models, and feature-based fusion
Case Study: Healthcare Analytics 8
 Healthcare analytics involves the systematic use of data analysis tools and
techniques to extract meaningful insights from healthcare data.
 It encompasses a wide range of applications aimed at improving patient
outcomes, enhancing operational efficiency, and optimizing decision-making
processes within the healthcare industry.
 Electronic Health Record (EHR) systems are used by almost 95% of doctors in
the US to gather, store, and analyse patient data.

Figure 3:
Big Data and
Healthcare
Case Study: Healthcare Analytics (contd..) 9
Applications:
 Predictive Analytics for Disease Prevention: It identifies patterns and risk factors,
allowing healthcare providers to proactively intervene and implement preventive measures.
 Personalized Medicine: By analyzing genetic, clinical, and lifestyle data, personalized
medicine ensures that treatment plans are optimized for the unique characteristics of each
patient, leading to more effective outcomes with fewer side effects.

Challenges:
 Data Privacy and Security
 Interoperability

Benefits:
 Improved Patient Outcomes
 Cost Reduction
Case Study: Supply Chain Optimization 10
 ML can be used in analyzing the big data of supply chain for demand forecasting and managing inventory
accordingly.
 It also includes using the transportation data like shortest routes , traffic, transport mode to optimize cost.
 Suppliers performance can also be measured and select accordingly to minimize supply disruptions.

Figure 4:
Uses of Big data and ML in
supply chain optimization
Case Study: Supply Chain Optimization 11

(contd..)
 ML can also provide insights of market from big data which will support decision
making.

 ML can also be used to warehouse layout data and optimize space, time and labor
cost.

 And in the end ML can also process customers reviews, product quality to ensure
customer satisfaction and quality control
Case Study : Human Resource & Talent Management 12
 Human Resource and Talent Management involves leveraging data analytics
to optimize the acquisition, development, and retention of talent within an organization.

 It encompasses a strategic approach to aligning the workforce with business goals and maximizing individual
and collective performance.

Applications:
 Recruitment Analytics: Utilizing data analytics in recruitment processes to identify and attract top talent.

Figure 5:
HR & Talent Management
Case Study : Human Resource & Talent Management(Cntd..)
Employee Engagement Analysis: 13
 Assessing employee engagement through surveys, feedback, and performance metrics.

 Data analytics helps identify factors influencing engagement, enabling proactive interventions
to enhance workplace satisfaction.

Talent Retention Strategies:

 Developing strategies to retain key talent by analyzing factors such as job satisfaction, career
progression, and workplace culture.

 Predictive analytics can identify employees at risk of leaving, allowing for targeted retention
efforts.

Challenges:
Data Quality and Accuracy
Ethical Considerations

Benefits:
Informed Decision-Making
Strategic Workforce Planning
Challenges and Considerations 14

 Maintaining the data quality is difficult as the amount of data is vast.

 Analyzing various data formats from various data source is challenging.

 Output generation rate is slower than big data streaming rate as input in ML
models creating high latency.

 Data security and privacy challenges as encrypting high amount of confidential

data is difficult.

 High dimensional feature space may cause overfitting.

Challenges and Considerations (contd..) 15

 High cost of setting up and maintaining of infrastructure.

 Difficulties in finding skilled experts needed for operating such complex models.

 Training the model with large complex data is also one of the major issues.

 The organizations using ML for big data analytics will have to comply with the
security regulations of government to avoid legal risks.
Future Trends
 Automated Machine Learning (AutoML): Increasing adoption of AutoML for simplifying and
16
automating the machine learning model development process, making it more accessible for non-experts.

 Explainable AI (XAI): Growing emphasis on Explainable AI to make machine learning models more
interpretable, transparent, and understandable for improved trust and accountability.

 Edge Computing Integration: Greater integration of machine learning models with edge computing,
enabling real-time analytics and reducing the need for centralized processing.

 Federated Learning: Rise of federated learning, allowing models to be trained across decentralized
devices without sharing raw data, enhancing privacy and scalability.

 Natural Language Processing : Advancements: Advancements in Natural Language Processing (NLP)

for better understanding and processing human language, leading to improved text and speech analytics.

 AI-Driven Automation for Data Management: Increasing use of AI-driven automation in data
management tasks, including data cleaning, integration, and governance, to enhance efficiency and reduce
manual efforts.
Conclusion 17

 Big Data and Machine learning work interdependently.

 Machine learning helps in processing the big data that is to be used

in various applications and sectors.

 Using Machine learning in Big data also comes with fair share of
challenges.

 Even so there are challenges , there is also a lot of scope to grow.

 18
References
1. https://www.javatpoint.com/what-is-big-data-and-machine-learning

2. https://www.geeksforgeeks.org/difference-between-big-data-and-machi
ne-learning/

3. https://chat.openai.com/
 19

THANK YOU