Learning Objectives What is Data Science?

o Introduction to data science Data science is the field of study that combines
o Domain expertise
o Relationship between data science and
o Programming skills
artificial intelligence o Knowledge of math and statistics
o Comprehend the process of data to extract meaningful insights from data

collection and generation

o Learn about various data categorization

What is Data Science? What is Data Science?

Multidisciplinary field! Data science practitioners apply
machine learning algorithms to
data such as numbers, text, images, video,
audio to
develop systems to perform tasks which
ordinarily require human intelligence.
Data Science and Machine Learning Data Science and Machine Learning
Amount of data is growing exponentially due “machine learning”
to collection and storage of digital data
● Computers automatically detect patterns
and make predictions or decisions from
data
● Learn from data without relying on a
predetermined mathematical model.
● Is a subset of Artificial Intelligence (AI).

Data Science and Machine Learning Data Science and Machine Learning
“machine learning” Machine learning systems
generate insights
● Computers automatically detect patterns
and make predictions or decisions from that analysts and business users
data translate into tangible
business value.
● Learn from data without relying on a
predetermined mathematical model.
● Is a subset of Artificial Intelligence (AI). Solving useful problems,
otherwise useless
Data Science and Machine Learning Applications of Data Science
Actual data science workflow can BUSINESS
be complex! ● Help businesses to increase business value of
its available data for competitive advantage
against their competitors.
● Understand customers better
● Take better decisions

Applications of Data Science Example Applications

SOCIAL GOOD Credit Card Fraud Detection

Applications in: ● Formulate a supervised model to

○ Agriculture categorize it into either fraud or no
○ Education fraud.
○ Disaster management
○ Environment ● Ideally, you would have a good
○ Transportation etc. quantity of examples.
Example Applications Example Applications
Credit Card Fraud Detection Features
Customer Segmentation
● monetary amount
● frequency ● Use of an unsupervised model
● Formulate a supervised model to
● place
categorize it into either fraud or no ● Find patterns about somebody
● period
fraud. who buys specific products.
● transaction information
● transaction class ● Build a targeted marketing
● Ideally, you would have a good campaign for these consumers.
quantity of examples. Labels
● fraud
● no fraud

Example Applications Roles in Data Science

Features
Customer Segmentation
● products purchased
● Use of an unsupervised model ● location
● Find patterns about somebody ● spending rate “Data Scientist is a person who is better at statistics than any
who buys specific products. ● product manufacturers programmer and better at programming than any
● education statistician.”
● Build a targeted marketing ● income
campaign for these consumers. ● age
-Josh Wills, Head of Data Engineering at Slack
Labels
● None
Roles in Data Science Roles in Data Science

Recap: What is Data Science? Recap: What is Data Science?

Data Analysis &

!Red flags:
Problem Formulation Presentation
Modeling
o Taking shortcuts: not spending
Data Collection &
Insight/Prediction
enough time with data and problem
Processing
but jumping to modeling
o Mindless use of ML tools
Skills: Tools: o Ethical breach
● Domain expertise ● Machine learning o New learners are mostly susceptible…
● Programming skills ● Statistical tools
● Mathematical foundation ● R/Python libraries

An overly simplified block diagram

Recap: What is Data Science? Data Generation
Source of data
Mindset:
○ Need to capture information of a
o Insights that are significant to the
problem physical/digital activities
o Figuring out the non-obvious! ○ E.g., Sales, customer feedback, social
o Data driven scientific mindset media posts, speech, temperature,
o Spend lot of time with exploring data body movement etc.
○ As you can see “everything under
the sun” and “flowing through the
internet”

Data Generation Data Gathering

Data gathering
○ Collected using sensors or manual labour or
mining data from web
○ Digital data collected by sensors
○ Manual annotation and data entry to
computer from physical documents
○ Extract data from internet using scripts
Data Generation Data Categories

The resulted data is raw data ● Structured versus unstructured data

○ With specific formats (.raw/..mp4 ● Quantitative versus qualitative data
for video, .wav for speech, .csv for
tabular data etc.)
○ Not clean and suitable for analysis
○ Need to clean: an important step in
data science workflow

Structured and Unstructured Data Structured and Unstructured Data

● Structured (organized) data: E.g.,
○ This is data that can be thought of as observations and
● Most data that exists in text form, including server logs and
characteristics. It is usually organized using a table method
Facebook posts, is unstructured
(rows and columns).
● Scientific observations, as recorded by careful scientists, are kept in
● Unstructured (unorganized) data: a very neat and organized (structured) format
○ This data exists as a free entity and does not follow any
standard organization hierarchy.
Structured and Unstructured Data Structured and Unstructured Data
● Data scientist likely prefers structured data
but they also must be able to deal with the
world's massive amounts of unstructured
data.
● If 80-90% of the world's data is unstructured
● Data pre-processing is used to apply
transformations to convert unstructured data
into a structured counterpart
Source: lawtomated.com

Structured and Unstructured Data Quantitative and Qualitative

Data pre-processing is used to apply transformations to convert ● Quantitative data: This data can be described
using numbers, and basic mathematical
unstructured data into a structured counterpart.
procedures, including addition, are possible on
the set.
Key characteristics:
“This Wednesday morn, are you early to rise? Then look East. The ● Qualitative data: This data cannot be
Crescent Moon joins Venus & Saturn. Afloat in the dawn skies.”
described using numbers and basic
mathematics. This data is generally thought of
as being described using natural categories
and language.
Quantitative and Qualitative Quantitative and Qualitative

● Name of a coffee shop: Qualitative

● Revenue: Quantitative
● Zip code: Qualitative
● Average monthly customers: Quantitative
● Country of coffee origin: Qualitative

Quantitative and Qualitative Quantitative and Qualitative

For a quantitative column, you may ask For a qualitative column, none of the
questions such as the following: preceding questions can be answered.
o What is the average value? However, the following questions only
o Does this quantity increase or apply to qualitative values:
decrease over time (if time is a factor)? o Which value occurs the most and the
o Is there a threshold where if this least?
number became too high or too low, it
o How many unique values are there?
would signal trouble for the company?
o What are these unique values?
Summary of the module
o What is data science?
o Data science vs. machine learning
o Example applications
o Roles in data science
o Data generation
o Categories of data