Scribd
Upload
11 views

Unit 1. Overview of Multivariate Methods Part 2

The document discusses the importance of validity and reliability in measurement, emphasizing that validity ensures a measure accurately reflects the intended concept, while reliability indicates consistency in measurement. It also highlights the value of multivariate measurement techniques, which combine multiple variables to provide a more comprehensive understanding of a concept, thereby reducing measurement error. Additionally, the document explains the relationship between statistical significance, power, and factors influencing power, including effect size, sample size, and alpha levels.

Uploaded by

Mirah Joy Delaspi - Hilaga
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
11 views

Unit 1. Overview of Multivariate Methods Part 2

The document discusses the importance of validity and reliability in measurement, emphasizing that validity ensures a measure accurately reflects the intended concept, while reliability indicates consistency in measurement. It also highlights the value of multivariate measurement techniques, which combine multiple variables to provide a more comprehensive understanding of a concept, thereby reducing measurement error. Additionally, the document explains the relationship between statistical significance, power, and factors influencing power, including effect size, sample size, and alpha levels.

Uploaded by

Mirah Joy Delaspi - Hilaga
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 21

b.

Measurement Error and Multivariate Measurement

VA L I D I T Y A N D R E L I A B I L I T Y
The researcher ’s goal of reducing measurement error
can follow several paths. In assessing the degree of
measurement error present in any measure, the
researcher must address two important characteristics
of a measure:
b. Measurement Error and Multivariate Measurement

Va l i d i t y i s t h e d e g r e e t o w h i c h a m e a s u r e a c c u r a t e l y
represents what it is supposed to. For example, if we
want to measure discretionary income, we should not
ask about total household income. Ensuring validity
starts with a thorough understanding of what is to be
measured and then making the measurement as
“correct” and accurate as possible.
b. Measurement Error and Multivariate Measurement

Reliability is the degree to which the observed


variable measures the “true” value and is “error free”;
t h u s , i t i s t h e o p p o s i t e o f m e a s u r e m e n t e r r o r. I f t h e
s a m e m e a s u r e i s a s k e d r e p e a t e d l y, f o r e x a m p l e , m o r e
reliable measures will show greater consistency than
less reliable measures.
b. Measurement Error and Multivariate Measurement

E M P L O Y I N G M U LT I VA R I AT E M E A S U R E M E N T
In addition to reducing measurement error by
improving individual variables, the researcher may
also choose to develop multivariate
measurements, also known as summated
scales, for which several variables are joined in a
composite measure to represent a concept (e.g.,
multiple-item personality scales or summed ratings of
product satisfaction).
b. Measurement Error and Multivariate Measurement

The objective is to avoid the use of only a single


variable to represent a concept and instead to use
several variables as indicators, all representing
differing facets of the concept to obtain a more well-
rounded perspective. The use of multiple indicators
enables the researcher to more precisely specify the
desired responses. It does not place total reliance on
a single response, but instead on the “average” or
typical response to a set of related responses.
b. Measurement Error and Multivariate Measurement

The impact of measurement error and poor reliability


cannot be directly seen because they are embedded
in the observed variables. The researcher must
therefore always work to increase reliability and
v a l i d i t y, w h i c h i n t u r n w i l l r e s u l t i n a m o r e a c c u r a t e
portrayal of the variables of interest.
b. Measurement Error and Multivariate Measurement

P o o r r e s u l t s a r e n o t a l w a y s d u e t o m e a s u r e m e n t e r r o r,
but the presence of measurement error is guaranteed
to distort the observed relationships and
make multivariate techniques less powerful. Reducing
m e a s u r e m e n t e r r o r, a l t h o u g h i t t a k e s e f f o r t , t i m e , a n d
additional resources, may improve weak or marginal
results and strengthen
proven results as well.
e . S TAT I S T I C A L S I G N I F I C A N C E V E R S U S S TAT I S T I C A L P O W E R

All the multivariate techniques, except for cluster


analysis and perceptual mapping, are based on the
statistical inference of a population’s values or
relationships among variables from a randomly drawn
sample of that population. A census of the entire
p o p u l a t i o n m a k e s s t a t i s t i c a l i n f e r e n c e u n n e c e s s a r y,
because any difference or relationship, however
small, is true and does exist. Researchers very
seldom use a census. Therefore, researchers are
often interested in drawing inferences from a sample.
e . S TAT I S T I C A L S I G N I F I C A N C E V E R S U S S TAT I S T I C A L P O W E R

By specifying an alpha level, the researcher sets the


acceptable limits for error and indicates the
probability of concluding that significance exists when
it really does not.
e . S TAT I S T I C A L S I G N I F I C A N C E V E R S U S S TAT I S T I C A L P O W E R

W h e n s p e c i f y i n g t h e l e v e l o f Ty p e I e r r o r, t h e
r e s e a r c h e r a l s o d e t e r m i n e s a n a s s o c i a t e d e r r o r,
t e r m e d Ty p e I I e r r o r, o r b e t a ( β ) . T h e Ty p e I I e r r o r i s
the probability of not rejecting the null
hypothesis when it is actually false. An extension of
Ty p e I I e r r o r i s 1 – β , r e f e r r e d t o a s t h e p o w e r
of the statistical inference test.
e . S TAT I S T I C A L S I G N I F I C A N C E V E R S U S S TAT I S T I C A L P O W E R

Power is the probability of correctly rejecting the null


hypothesis when it should be rejected. Thus, power is
the probability that statistical significance will be
indicated if it is present. The relationship of the
different error probabilities in testing for the
difference in two means is shown here:
e . S TAT I S T I C A L S I G N I F I C A N C E V E R S U S S TAT I S T I C A L P O W E R
e . S TAT I S T I C A L S I G N I F I C A N C E V E R S U S S TAT I S T I C A L P O W E R

Impacts on Statistical Power

But why can’t high levels of power always be


achieved? Power is not solely a function of alpha.

Power is determined by three factors:


e . S TAT I S T I C A L S I G N I F I C A N C E V E R S U S S TAT I S T I C A L P O W E R

Power is determined by three factors:

1. Effect size—The probability of achieving statistical


significance is based not only on statistical
considerations, but also on the actual size of the
effect. Thus, the effect size helps researchers
determine whether the observed relationship
(difference or correlation) is meaningful.
e . S TAT I S T I C A L S I G N I F I C A N C E V E R S U S S TAT I S T I C A L P O W E R

For example, the effect size could be a difference in


the means between two groups or
the correlation between variables. If a weight loss
firm claims its program leads to an average
weight loss of 25 pounds, the 25 pounds is the effect
size.
e . S TAT I S T I C A L S I G N I F I C A N C E V E R S U S S TAT I S T I C A L P O W E R

S i m i l a r l y, i f a u n i v e r s i t y c l a i m s i t s M B A g r a d u a t e s g e t
a starting salary that is 50 percent higher than the
average, the percent is the effect size attributed to
earning the degree. When examining effect sizes, a
larger effect is more likely to be found than a smaller
effect and is thus more likely to impact the power of
the statistical test.
e . S TAT I S T I C A L S I G N I F I C A N C E V E R S U S S TAT I S T I C A L P O W E R

3. Sample size—At any given alpha level, increased


sample sizes always produce greater power for the
statistical test. As sample sizes increase, researchers
must decide if the power is too high. By “too high” we
mean that by increasing sample size, smaller and
smaller effects (e.g., correlations) will be found to be
statistically significant, until at very large sample
sizes almost any effect is significant.
e . S TAT I S T I C A L S I G N I F I C A N C E V E R S U S S TAT I S T I C A L P O W E R

2. Alpha (a)—As alpha becomes more restrictive,


power decreases. Therefore, as the researcher
reduces the chance of incorrectly saying an effect is
significant when it is not, the probability of correctly
finding an effect decreases. Conventional guidelines
suggest alpha levels of .05 or .01. Researchers
should consider the impact of a particular alpha level
on the power before selecting the alpha level. The
relationship of these two probabilities is illustrated in
later discussions.
e . S TAT I S T I C A L S I G N I F I C A N C E V E R S U S S TAT I S T I C A L P O W E R

2. Alpha (a)—As alpha becomes more restrictive,


power decreases. Therefore, as the researcher
reduces the chance of incorrectly saying an effect is
significant when it is not, the probability of correctly
finding an effect decreases. Conventional guidelines
suggest alpha levels of .05 or .01. Researchers
should consider the impact of a particular alpha level
on the power before selecting the alpha level. The
relationship of these two probabilities is illustrated in
later discussions.
e . S TAT I S T I C A L S I G N I F I C A N C E V E R S U S S TAT I S T I C A L P O W E R

The researcher must always be aware that sample


size can affect the statistical test either by making it
insensitive (at small sample sizes) or overly sensi?
tive (at very large sample sizes).
e . S TAT I S T I C A L S I G N I F I C A N C E V E R S U S S TAT I S T I C A L P O W E R

You might also like