A

Micro-Project Report On

“Big Data”

Submitted By

Pathar Jinakal & 236200316158

Semester – 3rd Division – 3b

Department Of Information Technology Government

Polytechnic Rajkot

October - 2024

1
 CRETIFICATE

This is to certify that this micro-project report entitled

“BIG DATA” submitted by “Pathar Jinkal ” ,

“236200316158”, “ 3rd sem” has satisfactorily completed
her term work in the subject of SEMINAR.

Date : 14-9-2024 Guided By: Amit Bhalaodiya

Place : Rajkot

2
INDEX

 Introduction About Topic 4

 history of technology in Big Data And technology tools 5
 Working In Big Data 6
 Use Of Big Data 7
 Advantages In Big Data 8
 Limitation And Disadvantages 9
 Future Expansion In Big Data 10

3
Introduction About Topic
 Big data refers to extremely large datasets that are too complex or va
st for traditional data processing tools to handle.
 This data can come from various sources, like social media, sensors,
transactions, and more.
 The key characteristics of big data are often referred to as the three V
's: volume (the amount of data), velocity (the speed at which data is g
enerated), and variety (the different types of data).
 The goal of big data analytics is to extract meaningful insights and inf
ormation from these massive datasets.
 It involves using advanced tools and techniques like machine learnin
g, data mining, and statistical analysis to process and analyze the da
ta.

Here’s a quick intro to big data

 Volume: The sheer amount of data generated, measured in petabyte
s or exabytes.
 Velocity: The speed at which new data is created and moved.
 Variety: The different forms of data, ranging from structured to unstr
uctured.
 Value: The potential insights and benefits that can be extracted from
the data.
 Veracity: The accuracy and reliability of the data.

4
Here’s a brief history of technology in big data
 1940s: The concept of "information explosion" emerged, highlighting
the rapid growth of data1.
 1960s-
1970s: Mainframe computers were introduced, significantly increasi
ng data storage capacities2.
 1980s: The development of relational databases allowed for more eff
icient data organization and retrieval.
 1990s: The internet boom led to an exponential increase in data gene
ration and the need for better data management tools.
 2000s: Hadoop, an open-
source framework for distributed storage and processing of large dat
asets, was developed2.
 2010s: Big data analytics became mainstream, with advancements i
n machine learning, AI, and cloud computing.
 2020s: Real data processing and the Internet of Things (IoT) have furt
her expanded the scope and capabilities of big data technologies.

Here are some key big data technology tools

 Apache Hadoop: An open-
source framework for distributed storage and processing of large
datasets.
 Apache Spark: An open-source analytics engine for large-
scale data processing.
 Apache Flink: A stream processing framework for real-
time data processing.
 Google Cloud Platform: Provides various big data services, includin
g storage, analytics, and machine learning.
 MongoDB: A NoSQL database designed for handling large volumes o
f unstructured data.
 Sisense: A business intelligence tool that allows for data integration
and visualization.
 RapidMiner: A data science platform for data preparation, machine l
earning, and predictive analytics.

5
Working In Big Data
 Data Collection: Gathering large amounts of data from various sour
ces like social media, sensors, transactions, and more.
 Data Storage: Using scalable storage solutions like Hadoop Distribu
ted File System (HDFS) or cloud storage to store the vast data sets.
 Data Processing: Leveraging tools like Apache Spark, Hadoop MapR
educe, or real-
time processing frameworks like Apache Flink to process and manag
e the data.
 Data Analysis: Applying analytical techniques and machine learning
algorithms to uncover patterns, correlations, and insights. This can b
e done using tools like R, Python, or RapidMiner.
 Data Visualization: Presenting the data in an understandable format
using visualization tools like Tableau, Power BI, or Sisense. This help
s in making data-driven decisions.

These steps form a cycle where the insights gained can influence further da
ta collection and analysis, continuously improving the decision-
making process.

6
Use Of Big Data
1. Healthcare: Analyzing large datasets from electronic health records
to improve patient care, predict disease outbreaks, and optimize trea
tment plans.

2. Finance: Detecting fraud, managing risk, and guiding investment stra

tegies by analyzing transaction data and market trends.

3. Retail: Understanding customer behavior, personalizing marketing ef

forts, and optimizing supply chains by analyzing sales data and custo
mer feedback.

4. Telecommunications: Managing network performance, improving c

ustomer service, and predicting maintenance needs by analyzing usa
ge patterns and service data.

5. Transportation: Optimizing routes, managing fleets, and predicting

maintenance issues by analyzing vehicle data and traffic patterns.

6. Manufacturing: Enhancing production efficiency, improving product

quality, and predicting equipment failures by analyzing production da
ta and machine logs.

7. Public Sector: Enhancing public services, improving urban planning,

and managing emergencies by analyzing data from various public so
urces.

7
Advantages In Big Data
1. Better Decision Making: Data-
driven insights help organizations make more informed and accurate
decisions.

2. Enhanced Customer Experiences: Personalizing customer interacti

ons and offerings based on behavior and preferences.

3. Operational Efficiency: Streamlining processes and reducing costs

through data analysis and automation.

4. Innovation and Product Development: Identifying trends and custo

mer needs to create new products and services.

5. Predictive Analytics: Anticipating future trends and behaviors to sta

y ahead of the competition.

6. Risk Management: Detecting and mitigating risks more effectively by

analyzing patterns and anomalies

7. Competitive Advantage: Leveraging data to gain insights that comp

etitors may not have.

8
Limitation And Disadvantage
1. Data Quality: Ensuring the accuracy and reliability of massive datas
ets can be challenging.

2. Complexity: Handling and analyzing big data requires sophisticated

tools and expertise.

3. Privacy Concerns: Collecting and processing large amounts of data

can raise significant privacy and ethical issues.

4. Cost: Infrastructure and tools for storing and processing big data can
be expensive.

5. Data Overload: The sheer volume of data can make it difficult to extr
act meaningful insights without proper techniques.

6. Security Risks: Large datasets can be vulnerable to security breache

s and cyber-attacks.

7. Integration: Integrating big data with existing systems and workflows

can be complex and time-consuming.

9
Future Expansion in Big Data
1. Artificial Intelligence (AI) and Machine Learning (ML): AI and ML wil
l play an increasingly important role in big data analysis, helping busi
nesses quickly and accurately make sense of vast amounts of data1.

2. Edge Computing: Processing data closer to the source, rather than s

ending it to a centralized location, will reduce latency and improve re
al-time data analysis1.

3. Internet of Things (IoT): The proliferation of IoT devices will generate

even more data, providing deeper insights and enabling smarter deci
sion-making2.

4. Cloud Computing: Widespread migration to the cloud will offer scal

able and flexible solutions for storing and processing big data3.

5. Data Observability: Enhanced tools for monitoring and managing da

ta quality will ensure more reliable and actionable insights3.

6. Advanced Analytics: New techniques and tools will continue to evol

ve, allowing for more sophisticated analysis and predictive modeling.

10