UNIT –II BIG DATA

Big Data: Data Evolution - Terminologies - Definitions -Merits and Challenges - Big Data
Components-Characteristics - Big Data Processing Frameworks - Big Data Applications – Tools
for Big data Analytics.

What is Data?

Data is defined as individual facts, such as numbers, words, measurements, observations or just
descriptions of things.

For example, data might include individual prices, weights, addresses, ages, names, temperatures, dates,
or distances.

There are two main types of data:

1. Quantitative data is provided in numerical form, like the weight, volume, or cost of an item.

2. Qualitative data is descriptive, but non-numerical, like the name, sex, or eye colour of a person.

Characteristics of Data

The following are six key characteristics of data which discussed below:

Accuracy Data should be sufficiently accurate for the intended use and should
be captured only once, although it may have multiple uses.
Validity Data should be recorded and used in compliance with relevant
requirements, including the correct application of any rules or
definitions
Completeness Data requirements should be clearly specified based on the
information needs of the organization and data collection processes
matched to these requirements.
Reliability Data should reflect stable and consistent data collection processes
across collection points and over time.
Relevance Data captured should be relevant to the purposes for which it is to be
used.
Timeliness Data should be captured as quickly as possible after the event or
activity and must be available for the intended use within a
reasonable time period.

Types of Digital Data

Digital data is the electronic representation of information in a format or language that machines can
read and understand. In more technical terms, Digital data is a binary format of information that's
converted into a machine-readable digital format. The power of digital data is that any analog inputs,
from very simple text documents to genome sequencing results, can be represented with the binary
system.
 a) Structured
Structured is one of the types of big data and By structured data, we mean data that can be
processed, stored, and retrieved in a fixed format. It refers to highly organized information that
can be readily and seamlessly stored and accessed from a database by simple search engine
algorithms. For instance, the employee table in a company database will be structured as the
employee details, their job positions, their salaries, etc., will be present in an organized manner.
b) Unstructured
Unstructured data refers to the data that lacks any specific form or structure whatsoever. This
makes it very difficult and time-consuming to process and analyze unstructured data. Email is an
example of unstructured data.
c) Semi-structured
Semi structured is the third type of big data. Semi-structured data pertains to the data
containing both the formats mentioned above, that is, structured and unstructured data. To be
precise, it refers to the data that although has not been classified under a particular repository
(database), yet contains vital information or tags that segregate individual elements within the
data.

Big Data

Big Data is a collection of data that is huge in volume, yet growing exponentially with time. It is a data
with so large size and complexity that none of traditional data management tools can store it or process
it efficiently. Big data is also a data but with huge size.

Example:

Every day, 500+ terabytes of fresh data are absorbed into the Facebook systems. This information is
mostly gathered through photo and video uploads, message exchanges, and the posting of comments,
among other things. In 30 minutes of flying time, a single Jet engine may create 10+ gigabytes of data.
With thousands of flights every day, the amount of data generated can amount to several Petabytes.
Every day, the Fresh York Stock Exchange creates around a terabyte of new trading data

Big Data Characteristic

Volume: The name Big Data itself is related to an enormous size. Big Data is a vast ‘volume’ of data
generated from many sources daily, such as business processes, machines, social media platforms,
networks, human interactions, and many more.

Variety: Big Data can be structured, unstructured, and semi-structured that are being collected from
different sources. Data will only be collected from databases and sheets in the past, but these days the
data will comes in array forms, that are PDFs, Emails, audios, SM posts, photos, videos, etc.

Veracity Veracity means how much the data is reliable. It has many ways to filter or translate the data.
Veracity is the process of being able to handle and manage data efficiently. Big Data is also essential in
business development.
Value Value is an essential characteristic of big data. It is not the data that we process or store. It is
valuable and reliable data that we store, process, and also analyze.

Velocity Velocity plays an important role compared to others. Velocity creates the speed by which the
data is created in real-time. It contains the linking of incoming data sets speeds, rate of change, and
activity bursts.

The primary aspect of Big Data is to provide demanding data rapidly. Big data velocity deals with the
speed at the data flows from sources like application logs, business processes, networks, and social
media sites, sensors, mobile devices, etc.

Benefits of Big Data Processing

Ability to process Big Data brings in multiple benefits, such as-

1. Businesses can utilize outside intelligence while taking decisions.

2. Access to social data from search engines and sites like facebook, twitter are enabling organizations to
fine tune their business strategies.

. Improved customer service (Traditional customer feedback systems are getting replaced by new
systems designed with Big Data technologies.

4. Improved customer service (In these new systems, Big Data and natural language processing
technologies are being used to read and evaluate consumer responses.

5. Early identification of risk to the product/services, if any

Why is Big Data Important?

• Cost Savings

Big data helps in providing business intelligence that can reduce costs and improve the efficiency of
operations. Processes like quality assurance and testing can involve many complications particularly in
industries like biopharmaceuticals and nanotechnologies

• Time Reductions

Companies may collect data from a variety of sources using real-time in-memory analytics. Tools like
Hadoop enable businesses to evaluate data quickly, allowing them to make swift decisions based on
their findings.

• Understand the market conditions

Businesses can benefit from big data analysis by gaining a better grasp of market conditions. Analysing
client purchase behaviour, for example, enables businesses to discover the most popular items and
develop them appropriately. This allows businesses to stay ahead of the competition.

• Social Media

Listening’s Companies can perform sentiment analysis using Big Data tools. These enable them to get
feedback about their company, that is, who is saying what about the company. Companies can use Big
data tools to improve their online presence

• Using Big Data Analytics to Boost Customer Acquisition and Retention.

Customers are a crucial asset that each company relies on. Without a strong consumer base, no
company can be successful. However, even with a strong consumer base, businesses cannot ignore
market rivalry. It will be difficult for businesses to succeed if they do not understand what their
consumers desire. It will be difficult for businesses to succeed if they do not understand what their
consumers desire. It will result in a loss of customers, which will have a negative impact on business
growth. Businesses may use big data analytics to detect customer-related trends and patterns. Customer
behaviour

Using Big Data Analytics to Solve Advertisers Problem and Offer Marketing

analysis is the key to a successful business.

Insights All company activities are shaped by big data analytics. It allows businesses to meet client
expectations. Big data analytics aids in the modification of a company's product range. It guarantees that
marketing initiatives are effective.

• Big Data Analytics as a Driver of Innovations and Product Development