Chapter 1. Defining and Collecting Data Triệu Vi Gửi
Chapter 1. Defining and Collecting Data Triệu Vi Gửi
Topics covered:
Chapter 1: Defining and Collecting Data
Chapter 2: Organizing and Visualizing Variables
Chapter 3: Numerical Descriptive Measure
Chapter 4: Basic Probability
Chapter 5: Discrete Probability Distribution
Chapter 6: The Normal Distribution and Other Continuous Distributions
Chapter 7: Sampling Distribution
Chapter 8: Confidence Interval Estimation
Chapter 9: Fundamental of Hypothesis Testing: One-Sample Tests
Chapter 10: Two-Sample Tests
Chapter 11: Analysis of Variance
Chapter 13: Simple Linear Regression
Chapter 14: Introduction to Multiple Regression
May 10, 2023 1 / 43
Applied Statistics for Business
Chapter 1: DEFINING AND COLLECTING DATA
FPT University
Department of Mathematics
FUQN
Table of Contents
1 Defining Variables
2 Collecting Data
4 Data Cleaning
FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 3 / 43
Table of Contents
1 Defining Variables
2 Collecting Data
4 Data Cleaning
FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 4 / 43
Classifying Variables By Type
Variables
Categorical Numerical
FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 5 / 43
Categorical Variables
Categorical variables (qualitative variables) whose data represent categories.
→ This variables take values that are non-numerical in nature.
1 N ominal variable: describes a name, label or category without natural order.
2 Ordinal variable: whose values are defined by an order relation between the
different categories.
FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 8 / 43
Measurement Scales
Nominal Scale
A nominal scale classifies data into distinct categories in which no ranking is implied.
Ordinal Scale
An ordinal scale classifies data into distinct categories in which ranking is implied.
Ratio Scale
A ratio scale is an ordered scale in which the difference between the measurements is a
meaningful quantity and the measurements have a true zero point.
Quiz
Fill in the blank.
Data Scale, Type Values
Cellular provider ..., ... AT&T, T-Mobile, Verizon, Other, None
Excel skills ..., ... novice, intermediate, expert
Temperature (◦ F ) ..., ... −459.67◦ F or higher
SAT Math score ..., ... a value between 200 and 800, inclusive
Item cost (in $) ..., ... $0.00 or higher
FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 11 / 43
Bonus: For each of the following variables, determine whether the variable is categorical
or numerical and determine its measurement scale. If the variable is numerical, determine
whether the variable is discrete or continuous.
1 Number of cellphones in the household.
2 Monthly data usage (in MB).
3 Number of text messages exchanged per month.
4 Voice usage per month (in minutes).
5 Whether the cellphone is used for email.
FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 12 / 43
Table of Contents
1 Defining Variables
2 Collecting Data
4 Data Cleaning
FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 13 / 43
Population versus Sample
Sample
A sample contains only a portion of a population of interest.
Population Sample
Advertisements for IT jobs in the Vietnam The top 50 search results for advertise-
ments for IT jobs in the Vietnam on
May 1, 2022
Undergraduate students in the FPT University 200 undergraduate students majoring
in AI
All countries of the world Countries with published data available
on birth rates and GDP since 2010
FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 14 / 43
Question 1. A survey will be given to 100 students randomly selected from the freshmen
class at LQD High School. What is the population?
1 The 100 selected students.
2 All freshmen at LQD High School.
3 All students at LQD High School
Question 2. A survey will be given to 100 students randomly selected from the freshmen
class at LQD High School. What is the sample?
1 The 100 selected students.
2 All freshmen at LQD High School.
3 All students at LQD High School
Question 3. Fifty bottles of water were randomly selected from a large collection of
bottles in a company’s warehouse. These fifty bottles are referred to as the . . . . . .
1 population. 2 sample.
Question 4. Fifty bottles of water were randomly selected from a large collection of
bottles in a company’s warehouse. The large collection of bottles is referred to as the . . .
1 population. 2 sample.
FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 15 / 43
Note
1 In statistics, population can be a group of individuals, objects, events, organizations,
countries, species, organisms, etc.
2 The size of the sample is always less than the total size of the population.
Parameter vs Statistic
1 A parameter summarizes the value of a population for a specific variable.
2 A statistic summarizes the value of a specific variable for sample data.
Question 1. A survey of 2000 American households found that 33% of the respondents
own a computer. Is this value a parameter or a statistic?
Question 2. The average salary of all automotive workers is $42, 000. Is this value a
parameter or a statistic? FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 16 / 43
Data Sources
Data sources arise from the following activities:
Capturing data generated by ongoing business activities.
Distributing data compiled by an organization or individual.
Compiling the responses from a survey.
Conducting a designed experiment and recording the outcomes of the experiment.
Conducting an observational study and recording the results of the study.
FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 17 / 43
Examples of Data Collected From Ongoing Business Activities
A bank studies years of financial transactions to help them identify patterns of fraud.
Economists utilize data on searches done via Google to help forecast future
economic conditions.
Marketing companies use tracking data to evaluate the effectiveness of a web site.
FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 18 / 43
Examples of Data From A Designed Experiment
Consumer testing of different versions of a product to help determine which product
should be pursued further.
Material testing to determine which supplier’s material should be used in a product.
Market testing on alternative product promotions to determine which promotion to
use more broadly.
FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 19 / 43
Observational Studies & Designed Experiments
1 In both, researchers that collect data are looking for the effect of some change,
called a treatment, on a variable of interest.
2 In an observational study, the researcher collects data in a natural or neutral setting
and has no direct control of the treatment.
3 In a designed experiment, there is direct control over which items receive the
treatment.
1
Hint. Population: Return on all the IPOs in the US. Sample: Return on 250 IPOs in the US. FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 20 / 43
Table of Contents
1 Defining Variables
2 Collecting Data
4 Data Cleaning
FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 21 / 43
A Sampling Process Begins with a Sampling Frame
FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 22 / 43
Types of Samples
Samples
Systematic Cluster
FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 23 / 43
Non Probability Sample
In a non probability sample, items included are chosen without regard to their probability
of occurrence.
In a judgment sample, you get the opinions of pre-selected experts in the subject
matter.
In convenience sampling, items are selected based only on the fact that they are
easy, inexpensive, or convenient to sample.
Probability Sample
In a probability sample, items in the sample are chosen on the basis of known
probabilities.
Probability Samples
FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 24 / 43
Probability Sample: Simple Random Sample
Every individual or item from the frame has an equal chance of being selected.
Selection may be with replacement (selected individual is returned to frame for
possible reselection) or without replacement (selected individual is not returned to
the frame).
Samples obtained from table of random numbers or computer random number
generators.
FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 25 / 43
Steps to Conduct Simple Random Sampling
1 Make a list of all the members in the population (population size: N ).
2 Assign a sequential number to each member (i.e., 1, 2, . . . , N ).
→ This is the sampling frame (the list from which you draw your sample).
3 Figure out what the sample size is going to be (sample size: n).
4 Use a random number generator to select the sample.
→ For instance, if the sample size is 4 and the population is 12, generate 4 random
numbers between 1 and 12.
3
Source: Analytics Vidhya FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 27 / 43
Probability Sample: Stratified Sample
Divide population into two or more subgroups (called strata) according to some
common characteristic.
A simple random sample is selected from each subgroup, with sample sizes
proportional to strata sizes.
Samples from subgroups are combined into one.
This method is typically used when a population has distinct differences, such as
demographics, level of education, or age, and can easily be broken into subgroups.
Question. For a population containing N = 902 individuals, what code number would
you assign for
1 the first person on the list?
2 the fortieth person on the list?
3 the last person on the list?
FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 30 / 43
Table of Contents
1 Defining Variables
2 Collecting Data
4 Data Cleaning
FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 31 / 43
Data Cleaning Is An Important Data Preprocessing Task Prior To Analysis
FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 32 / 43
Examples (Coding Errors)
1 Copy-and-paste or data import can result in poor recording or entry of data.
2 Categorical variable: Gender, Correct coding: F or M.
Correctable error: Female.
Invalid data: New York.
Correctable or software tolerated: m.
Extraneous and nonprintable characters: Leading or trailing space(s) including _F or
F_, other nonprintable characters may also be leading or trailing.
FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 33 / 43
Missing Values
1 Missing values are values that were not collected for a variable.
2 For example, survey data may include answers for which no response was given by
the survey taker. Such “no responses” are examples of missing values.
FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 34 / 43
Data Cleaning Cannot Be A Fully Automated Process
Excel, JMP, and Minitab have functionality to lessen the burden of data cleaning.
The software guides in the book explain this functionality.
When performing data cleaning, always preserve a copy of the original data for later
reference.
FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 35 / 43
Table of Contents
1 Defining Variables
2 Collecting Data
4 Data Cleaning
FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 36 / 43
Data Can Be Formatted and/or Encoded In More Than One Way
Some electronic formats are more readily usable than others.
Different encodings can impact the precision of numerical variables and can also
impact data compatibility.
As you identify and choose sources of data you need to consider or deal with these
issues.
1 Defining Variables
2 Collecting Data
4 Data Cleaning
FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 38 / 43
Evaluating Survey Worthiness
What is the purpose of the survey?
Is the survey based on a probability sample?
Coverage error - appropriate frame?
Nonresponse error - follow up.
Measurement error - good questions elicit good responses.
Sampling error - always exists.
FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 39 / 43
Types of Survey Errors
1 Coverage error or selection bias: Exists if some groups are excluded from the
frame and have no chance of being selected.
→ Excluded from frame.
2 Nonresponse error or bias: People who do not respond may be different from those
who do respond.
→ Follow up on nonresponses.
3 Sampling error: Variation from sample to sample will always exist.
→ Random differences from sample to sample.
4 Measurement error: Due to weaknesses in question design and/or respondent error.
→ Bad or leading question.
FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 40 / 43
Ethical Issues About Surveys
Coverage error and nonresponse error can be leveraged by survey designers to
purposely bias survey results.
Sampling error can be an ethical issue if the findings are purposely not reported with
the associated margin of error.
Measurement error can be an ethical issue:
Survey sponsor chooses leading questions.
Interviewer purposely leads respondents in a particular direction.
Respondent(s) willfully provide false information.
FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 41 / 43
Chapter Summary
FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 42 / 43
Thank you!
FUQN
VyNHT – FUQN MAS202 - Chapter 1 Quy Nhon, 2023 43 / 43