BIG DATA

What is big data

Big Data refers to complex and large data sets that have to be processed and analyzed to
uncover valuable information that can benefit businesses and organizations.
1. It refers to a massive amount of data that keeps on growing exponentially with time
2. It is so voluminous that it cannot be processed or analyzed using conventional data
processing techniques.
3. It includes data mining, data storage, data analysis, data sharing, and data visualization.

4. The term is an all-comprehensive one including data, data frameworks, along with the tools

and techniques used to process and analyze the data.

 TYPES Of big data
structured
Structured is one of the types of big data and By structured semi-structured
data, we mean data that can be processed, stored, and
retrieved in a fixed format. It refers to highly organized Semi structured is the third type of big data.
information that can be readily and seamlessly stored and Semi-structured data pertains to the data
accessed from a database by simple search engine algorithms. containing both the formats mentioned above,
For instance, the employee table in a company database will be that is, structured and unstructured data. To be
structured as the employee details, their job positions, their precise, it refers to the data that although has not
salaries, etc., will be present in an organized manner. been classified under a particular repository
(database), yet contains vital information or tags
unstructured that segregate individual elements within the
data. Thus we come to the end of types of data.
Let's discuss the characteristics of data.
Unstructured data refers to the data that lacks any specific
form or structure whatsoever. This makes it very difficult and
time-consuming to process and analyze unstructured data.
Email is an example of unstructured data. Structured and
unstructured are two important types of big data.
 CHARACTERISTICs of BIG DATA
variety
Variety of Big Data refers to structured, unstructured, and volume
semistructured data that is gathered from multiple sources. While
in the past, data could only be collected from spreadsheets and Volume is one of the characteristics of big data.
databases, today data comes in an array of forms such as We already know that Big Data indicates huge
emails, PDFs, photos, videos, audios, SM posts, and so much ‘volumes’ of data that is being generated on a
more. Variety is one of the important characteristics of big data. daily basis from various sources like social media
platforms, business processes, machines,
networks, human interactions, etc. Such a large
amount of data are stored in data warehouses.
Thus comes to the end of characteristics of big
velocity data.
Velocity essentially refers to the speed at which data is being
created in real-time. In a broader prospect, it comprises the
rate of change, linking of incoming data sets at varying
speeds, and activity bursts.
Advantages of big data
● One of the biggest advantages of Big Data is predictive analysis. Big Data analytics tools can predict
outcomes accurately, thereby, allowing businesses and organizations to make better decisions, while
simultaneously optimizing their operational efficiencies and reducing risks.

● By harnessing data from social media platforms using Big Data analytics tools, businesses around the
world are streamlining their digital marketing strategies to enhance the overall consumer experience. Big
Data provides insights into the customer pain points and allows companies to improve upon their
products and services.

● Being accurate, Big Data combines relevant data from multiple sources to produce highly actionable
insights. Almost 43% of companies lack the necessary tools to filter out irrelevant data, which eventually
costs them millions of dollars to hash out useful data from the bulk. Big Data tools can help reduce this,
saving you both time and money.

● Big Data analytics could help companies generate more sales leads which would naturally mean a boost
in revenue. Businesses are using Big Data analytics tools to understand how well their products/services
are doing in the market and how the customers are responding to them. Thus, the can understand better
where to invest their time and money.

● With Big Data insights, you can always stay a step ahead of your competitors. You can screen the market
to know what kind of promotions and offers your rivals are providing, and then you can come up with
better offers for your customers. Also, Big Data insights allow you to learn customer behavior to
understand the customer trends and provide a highly ‘personalized’ experience to them.
 WHo is using big data
1. Healthcare-Big Data has already started to create a huge difference in the healthcare sector. With the
help of predictive analytics, medical professionals and HCPs are now able to provide personalized
healthcare services to individual patients. Apart from that, fitness wearables, telemedicine, remote
monitoring – all powered by Big Data and AI – are helping change lives for the better.

2. Academia-Big Data is also helping enhance education today. Education is no more limited to the physical
bounds of the classroom – there are numerous online educational courses to learn from. Academic
institutions are investing in digital courses powered by Big Data technologies to aid the all-round
development of budding learners.

3. Banking-The banking sector relies on Big Data for fraud detection. Big Data tools can efficiently detect
fraudulent acts in real-time such as misuse of credit/debit cards, archival of inspection tracks, faulty
alteration in customer stats, etc.

4. manufacturing-According to TCS Global Trend Study, the most significant benefit of Big Data in
manufacturing is improving the supply strategies and product quality. In the manufacturing sector, Big
data helps create a transparent infrastructure, thereby, predicting uncertainties and incompetencies that
can affect the business adversely.

5. IT-One of the largest users of Big Data, IT companies around the world are using Big Data to optimize their
functioning, enhance employee productivity, and minimize risks in business operations. By combining Big Data
technologies with ML and AI, the IT sector is continually powering innovation to find solutions even for the most
complex of problems.
 BIg data case studies
1. Walmart- Walmart leverages Big Data and Data Mining to create personalized product recommendations for its
customers. With the help of these two emerging technologies, Walmart can uncover valuable patterns showing the most
frequently bought products, most popular products, and even the most popular product bundles (products that
complement each other and are usually purchased together).Based on these insights, Walmart creates attractive and
customized recommendations for individual users. By effectively implementing Data Mining techniques, the retail giant
has successfully increased the conversion rates and improved its customer service substantially. Furthermore, Walmart
uses Hadoop and NoSQL technologies to allow customers to access real-time data accumulated from disparate sources.

2. American express-The credit card giant leverages enormous volumes of customer data to identify indicators that could
depict user loyalty. It also uses Big Data to build advanced predictive models for analyzing historical transactions along
with 115 different variables to predict potential customer churn. Thanks to Big Data solutions and tools, American Express
can identify 24% of the accounts that are highly likely to close in the upcoming four to five months.

3. Uber- Uber is one of the major cab service providers in the world. It leverages customer data to track and identify the
most popular and most used services by the users. Once this data is collected, Uber uses data analytics to analyze the
usage patterns of customers and determine which services should be given more emphasis and importance.Apart from
this, Uber uses Big Data in another unique way. Uber closely studies the demand and supply of its services and changes
the cab fares accordingly. It is the surge pricing mechanism that works something like this – suppose when you are in a
hurry, and you have to book a cab from a crowded location, Uber will charge you double the normal amount!

4.
1. Procter and gamble-Procter & Gamble has been around us for ages now. However, despite being an “old” company,
P&G is nowhere close to old in its ways. Recognizing the potential of Big Data, P&G started implementing Big Data tools
and technologies in each of its business units all over the world. The company’s primary focus behind using Big Data
was to utilize real-time insights to drive smarter decision making.To accomplish this goal, P&G started collecting vast
amounts of structured and unstructured data across R&D, supply chain, customer-facing operations, and customer
interactions, both from company repositories and online sources. The global brand has even developed Big Data
systems and processes to allow managers to access the latest industry data and analytics.

2. IRS-Yes, even government agencies are not shying away from using Big Data. The US Internal Revenue Service actively
uses Big Data to prevent identity theft, fraud, and untimely payments (people who should pay taxes but don’t pay them
in due time).The IRS even harnesses the power of Big Data to ensure and enforce compliance with tax rules and laws. As
of now, the IRS has successfully averted fraud and scams involving billions of dollars, especially in the case of identity
theft. In the past three years, it has also recovered over US$ 2 billion.

3. Netflix-Netflix is one of the most popular on-demand online video content streaming platform used by people around
the world. Netflix is a major proponent of the recommendation engine. It collects customer data to understand the
specific needs, preferences, and taste patterns of users. Then it uses this data to predict what individual users will like
and create personalized content recommendation lists for them.Today, Netflix has become so vast that it is even
creating unique content for users. Data is the secret ingredient that fuels both its recommendation engines and new
content decisions. The most pivotal data points used by Netflix include titles that users watch, user ratings, genres
preferred, and how often users stop the playback, to name a few. Hadoop, Hive, and Pig are the three core components
of the data structure used by Netflix.
 Challenges of big data
1. Lack of proper understanding of big data-Companies fail in their Big Data initiatives due to insufficient
understanding. Employees may not know what data is, its storage, processing, importance, and sources.
Data professionals may know what is going on, but others may not have a clear picture.
2. Data growth issue-One of the most pressing challenges of Big Data is storing all these huge sets of data
properly. The amount of data being stored in data centers and databases of companies is increasing
rapidly. As these data sets grow exponentially with time, it gets extremely difficult to handle.Most of the data
is unstructured and comes from documents, videos, audios, text files and other sources. This means that
you cannot find them in databases.
3. Lack of data professionals-To run these modern technologies and Big Data tools, companies need skilled
data professionals. These professionals will include data scientists, data analysts and data engineers who
are experienced in working with the tools and making sense out of huge data sets.Companies face a
problem of lack of Big Data professionals. This is because data handling tools have evolved rapidly, but in
most cases, the professionals have not. Actionable steps need to be taken in order to bridge this gap.
4. Securing data-Securing these huge sets of data is one of the daunting challenges of Big Data. Often
companies are so busy in understanding, storing and analyzing their data sets that they push data security
for later stages. But, this is not a smart move as unprotected data repositories can become breeding
grounds for malicious hackers.Companies can lose up to $3.7 million for a stolen record or a data breach.
5.