UNIT-I

Research Problem
It refers to some difficulty which a researcher experiences in
the context of either theoretical or practical situation and wants
to obtain a solution for the same.
A research problem can be simply defined as a statement that
identifies the problem or situation to be studied.

Components of research problem

An individual or a group with some difficulty or problem

Objectives of research that are to be attained
The environment

 An individual or a group with some difficulty or problem.

 Objectives of research that are to be attained.
 The environment in which the problem exists.
action or Alternative means for obtaining the objective
Outcome
action or Alternative means for obtaining the objectiv
 Two or more course of action or Alternative means for
obtaining the objective
 Two or more possible Outcomes
 Objective of the study

Characteristics of a good topic?

 Interest-The Topic must be able to keep the researcher

interested in it throughout the research process.
 Data Availability – It must be ensured that the topic can be
investigated through the collection and analysis of data
 Significant-the topic must contribute towards
improvement and understanding of an educational theory
or practice
 Adequate-The topic must be according to the skills of the
researcher, available resources and time restrictions
 Ethical- The Topic must not embarrass or harm the
society,
Interest – The topic must be able to keep the researcher
interested in it throughout
Ethical – The topic must not embarrass or harm the society
  Selecting a problem
 Formulating a Research problem
 Necessity of defining a problem
 Technique involved in defining a research problem

A researcher may define a research problem by: -

 Sources of research
problem

II. Sources of Problems to Solve

Beginners tend to start with relatively specific research

problems focused on the face value of the question, but
eventually develop a broad research question with great
generality. For example, what started as “how can I help my
roommate study more?” evolves into “what controls studying
in people?” At the beginning, the roommate’s behaviour is
at issue for
itself. Later the person and the behaviour are seen as arbitrary
instances of a much more
important and challenging question. Career long research
problems tend to emerge following several years of specific
research topics, and require many specific research studies
to solve. This section details some of the sources for an
initial, relatively specific, research problem. It is intended to
help you come up with research which is manageable,
enjoyable, and productive.

A very serious impediment facing new researchers is well

illustrated by trying to use a foreign language dictionary to
learn what foreign words mean. Until you know "enough" of a
language, you cannot find out what the words mean. Until you
know "enough" of a paradigm, you do not know what
unresolved questions remain, or when the paradigm is wrong.
"A"
below is generally a person's first exposure to a research
project for that reason.

In addition to not knowing what unresolved problems

remain, is missing the more fundamental broader issue
underlying any specific behaviour change. When looking at the
world, try to see each functional relationship as only an
instance of a more general class of relationships.

A. Research Problem from Expert

The simplest source of a problem to solve is to have it given to
you as a class assignment, as a directed research project, or as
a task while you are an apprentice in someone's lab. You are
told what problem to research and how to do it. This is probably
an ideal way to assure that your first research topic is a good
one.

Example: Students in Experimental Psychology were assigned

the task of finding out if social attention made their
roommate study more. They were told to measure the amount
of time their roommate studied on days during which they
expressed interest in their roommate's course material as
compared to days when they refrained from talking about
academic topics.

B. Research Problem from Folklore

Common beliefs, common sense, or proverbs could be right but
on the other hand, they could also be wrong. You must verify
that they are true before considering them as a source of
knowledge. It is possible that some unverified beliefs have the
roots of a better idea and therefore would be a worthy research
topic. It is critical to note, however, that the task of research is
not to simply validate or invalidate common sense but rather to
come to understand nature.

Example: It's commonly believed that studying within the two

hours preceding a test will decrease test scores. To research
this belief a randomly selected half of a class was told to study
immediately before taking a test while the other half was
prohibited from studying before the test. This research was
intended to determine whether or not studying
immediately before a test decreased the points earned.

C. Research Problem from Insight

Sometimes people research an issue simply because it
occurred to them and it seemed
important. The systematic development of the idea is lacking.
This is "intuitive" or good

guess research. It is risky because you may not be able

to get other researchers to
understand why the research is important. It is fun because you
get to do what interests
you at the moment. Alternatively, it could be the application of
a general rule of thumb or
guessing that a new problem is actually a well-understood
function in disguise.
Example: While feeling especially competent after explaining
course material to three friends you realize that orally
presenting material may help test performance. You
conducted a study in which material was orally presented
before the test on a random half of the occasions. The
research was based on your insightful realization that oral
presentation may increase test performance.

D. Research Problem from Informal Discussion

This is a research problem that some discussion group
feels is interesting. Discussion among friends can often
spark our interest in a problem or provides us with the
reinforcers for pursuing a question.

Example: After telling a group of friends about your success

with oral presentations on test taking, the group talks about it
for a while and becomes interested in the possibility of the
subject becoming confused as well as doing better as a result
of feedback from the listeners. The group provides you with
the idea and the excitement to do research on how students
can affect the accuracy of a teacher's understanding.

E. Research Problem from Knowledge of Techniques and

Apparatus
This is the selection of a research topic based on your special
knowledge outside the field. A technique or apparatus with
which you are familiar can offer the potential for a major
advance in the field of psychology. Sometimes we realize
that we can apply a new technique or apparatus to an area
to which it has not yet been applied. Because we are specially
qualified to succeed, solving the problem can be especially
gratifying.

Example: You may know about microelectronics and be good at

detailed work. You find out that many researchers are anxious
to discover the migration patterns of butterflies so you mount
an integrated circuit transmitter on a butterfly and thereby
trace the behaviour of the free ranging butterfly.

F. Research Problem from Reading the Literature

These are research problems which capture your interest
while reading. While reading you will often wonder why, or
will disagree, or will realize that you have a better idea than
the original author.

Example: While you were reading about jet lag and its effects
on sleep the first night, you realize that the author failed to
control for light cycle. You try stretching either the light period
or stretching the dark period to make up the phase shift. You
implement this by changing the cabin illumination period on
various trans-Atlantic flights, and monitoring
the passengers sleep for the next three days.

1. Sources of Research Literature

Initially, it may be hard to know where to start reading on your
quest for knowledge. Consider starting at a very broad
general level and working your way to the more recondite.

A good place to start is several Introductory Psychology

textbooks. Understand the basic area, what it is, how it
fits in Psychology, and why it is important. Then look
through several middle level textbooks which cover that
particular area.
Understanding the structure of the area by reading the table of
contexts, then read the specific section relevant to your
research topic. Read the table of contents and sections





relevant to your paper in several more textbooks, paying
particular attention to the original research which led to the
general paradigm and conclusions. Pay attention to
the theoretical significance of various types of results and to
the functional relationship depicted in the figures. Note the
authors, titles, dates, volume, and pages of the journals and
books which are referred to. Then consult Psychological
Abstracts. This is a publication that organizes and provides
short abstracts of the mass of knowledge provided in
journals. The Psychological Abstracts are available in the
library. Locate
key papers in the Abstracts.
Introductory texts
Second-level or area texts
Annual Review / review articles
Special topic text / symposium reports
Journal articles
2. How to Find Additional Sources
Web
Psych Info
Dictionary of Psychology (http://www.psychology.org)
Card catalog
Psychological Abstracts
Current Contents
Citation Index (find subsequently-published related articles)
Reference sections of relevant papers (find previously-
published related articles)
Knowledgeable people
3. How to Read Research Articles
Actively participate while you are reading. At first it will keep
you from falling asleep, later it will keep you from thinking
about other things, and eventually it will make it a lot of fun.

Underline, write questions and answers in the margins, and

keep an idea log. Draw a diagram of the procedure. Consider
how the research bears on your interests. Look for what's
important.

a. What was the research problem and why must it be

answered?
b. What subjects were used and why?
c. What apparatus or setting was used and why?
d. What general procedure was used and why?
e. Was the procedure applicable and the best available?
f. What was the independent variable, how was it measured,
and what was it inferred
to be doing (its interpretation) (e.g., did shock produce fear or
something else)?
g. What were potential confounds and how were they
controlled?
h. What was the dependent variable, how was it measured, and
what was it inferred
to be the result of, or what did it represent (its interpretation)?
i. What were the actual results, what were they
interpreted to mean, and to what
extent is it likely they would happen again if the experiment
were replicated (their
reliability)?
j. How sensitive was the dependent measure? To what degree
would small changes in
the independent variable be expected to change the dependent
variable?
k. How much of the variability obtained could be accounted for?
l. To what extent will the findings apply to other subjects,
situations, and procedures?
Was there generality?
m. What was gained by the research ("so what")? How
has the paradigm been
extended by this finding?
G. Research Problem from a Paradoxical Incident or
Conflicting Results
If the world is perfectly understood, then there can be no
surprises. Contrariwise,if something surprises you, then your
theoretical framework is inadequate and needs
development. If two seemingly similar procedures produce
different results, then something is wrong with your
understanding of the procedures. They are not actually
similar in the important respect of how they affect the
dependent variable. Given that an
error has been made, something is not correctly understood
and must be resolved.

H. Research Problem Deduced from Paradigms or

Theories
Researchers who propose theoretical accounts for
phenomena cannot think through every possible
ramification. As you come to understand a theory,
potential errors or extensions become apparent. This type of
research tests the implications of theories to confirm or reject
them. This is classic deductive "normal" science. Using the
object in the lake from the first chapter as an example --
this would be deducing "if it is an steam shovel under
there, then we should find a long row of high spots coming out
of one end." You then test that prediction by probing around
trying to find a boom.
 Criteria Characteristics of a good research
problem

There are some suggestion for the graduate

students and researchers which are drawn from the different
areas of education, social sciences as well as psychology. There
are two factors in the selection of topic external and personal.
External criteria involve how the topic is important for the field,
availability of both data and data collection methods and the
administration is cooperative or not. Personal Criteria means
researcher own interest, time and cost. Criteria for selection of
research problem depends on the following characteristics.

Personal Inclination. The chief motivation in the way of

selecting research problem is the personal inclination of the
researcher. If a researcher has personal interest in the topic, he
would select that problem for his research work.

Resources Availability. During the selection, a researcher

will see to the resources available. If these resources like
money, time, accommodation and transport are available to the
selection place, then the selection of the problem is easy.

Relative Importance. The importance and the problem also

play a vital role in the selection of research problem. If the
problem is relatively important, then the researcher tends
towards the selection of the problem.

Researcher Knowledge. The researcher knowledge should

play a vital role in the selection of the research problem. The
wisdom and experience of an investigator is required for well
collection of the research data. He can bitterly select a
problem.

Practicality: Practicality is also responsible for the selection.

The practical usefulness of the problem is the main motivation
for a researcher to attend it.

Time-lines of the Problem. Some problems take little time for its
solution while others take more time. So, it depends on the
time in which we have to complete his research work.

Data Availability. If the desired data is available to the

researcher, then the problem would be selected.
Urgency. Urgency is a pinpoint in the way of the
selection of research problem. Urgent problem must be given
priority because the immediate solution can benefit the people.

Feasibility. Feasibility is also an important factor for the

selection of the research problem.
The researcher qualification, training and experience should
match the problem.

Area Culture. The culture of the area for which a researcher

conducts his research is also
responsible for the selection of research problem.

Characteristic of Research Problem

Any research is a difficult task to achieve and research needs to
do a great effort. Selection of research topic is the first step to
success.
1.Research topic must be very clear and easy to understand. It
should not distract people.
2.If a topic is well define is the only way to successful research.
The topic should not create doubt and double impression.
3.Easy language is a key to success. Use technical words if
necessary otherwise focus of simplicity.
4.Research title should be according to the rules of titling.
There are different rules of titling, a researcher must aware
before writing a research title.
5.While selecting a research topic current importance of a
researcher should also be considered. Topic should not be
obsolete and it should have great importance in the
current day.

Errors in selecting a research problem

Designing a research project takes time, skill and

knowledge. If you don’t go into the process with a clear goal
and methods, you’ll likely come out with skewed data or
an inaccurate picture of what you were trying to accomplish.
With Qualtrics survey software, we make the survey creation
process easier, but still you may feel overwhelmed with the
scope of your research project.
While it’s important to use proper methodology in the research
process, it’s equally important to avoid making critical
mistakes that could produce inaccurate results. In this
article, we’ll list 5 common errors in the research process and
tell you how to avoid making them, so you can get the best
data possible.

1.Population Specification

Population specification errors occur when the researcher

does not understand who they should survey. This can be
tricky because there are multiple people who might consume
the product, but only one who purchases it, or they may miss a
segment looking to purchase in the future.
Example: Packaged goods manufacturers often conduct
surveys of housewives, because they are easier to contact,
and it is assumed they decide what is to be purchased and also
do the actual purchasing. In this situation there often is
population specification error. The husband may purchase a
significant share of the packaged goods, and have significant
direct and indirect influence over what is bought. For this
reason, excluding husbands from samples may yield results
targeted to the wrong audience.
How to avoid this: Understand who purchases your product
and why they buy it. It’s important to survey the one making
the buying decision so you know how to better reach them.

2.Sampling and Sample Frame Errors

Survey sampling and sample frame errors occur when the

wrong subpopulation is used to select a sample, or because of
variation in the number or representativeness of the sample
that responds, but the resulting sample is not representative of
the population concern. Unfortunately, some element of
sampling error is unavoidable, but sometimes, it can be
predicted. For instance, in the 1936 presidential election
between Roosevelt and Landon, the sample frame was from car
registrations and telephone directories. The researchers failed
to realize that the majority of people that owned cars and
telephones were Republicans, and wrongly predicted a
Republican victory.
Example: Suppose that we collected a random sample of 500
people from the general U.S. adult population to gauge their
entertainment preferences. Then, upon analysis, found it to be
composed of 70% females. This sample would not be
representative of the general adult population and would
influence the data. The entertainment preferences of females
would hold more weight, preventing accurate extrapolation
to the US general adult population. Sampling error is
affected by the homogeneity of the population being studied
and sampled from and by the size of the sample.
How to avoid this: While this cannot be completely
avoided, you should have multiple people reviewing your
sample to account for an accurate representation of your
target population. You can also increase the size of your
sample so you get more survey participants.

3.Selection

Selection error is the sampling error for a sample selected

by a non-probability method. When respondents choose to
self-participate in a study and only those interested respond,
you can end up with selection error because there may already
be an inherent bias. This can also occur when respondents who
are not relevant to the study participate, or when there’s a bias
in the way participants are put into groups.
Example: Interviewers conducting a mall intercept study have
a natural tendency to select those respondents who are the
most accessible and agreeable whenever there is latitude to do
so. Such samples often comprise friends and associates
who bear some degree of resemblance in characteristics to
those of the desired population.
How to avoid this: Selection error can be controlled by
going extra lengths to get participation. A typical survey
process includes initiating pre-survey contact requesting
cooperation, actual surveying, and post-survey follow-up. If
a response is not received, a second survey request
follows, and perhaps interviews using alternate modes
such as
telephone or person-to-person.

4.Non-responsive
Nonresponse error can exist when an obtained sample
differs from the original selected sample. This may occur
because either the potential respondent was not contacted or
they refused to respond. The key factor is the absence of data
rather than inaccurate data.
Example: In telephone surveys, some respondents are
inaccessible because they are not at home for the initial call
or call-backs. Others have moved or are away from home
for the period of the survey. Not-at-home respondents are
typically younger with no small children, and have a much
higher proportion of working wives than households with
someone at
home. People who have moved or are away for the survey
period have a higher geographic
mobility than the average of the population. Thus, most
surveys can anticipate errors from non-contact of
respondents. Online surveys seek to avoid this error
through e-mail distribution, thus eliminating not-at-home
respondents.
How to avoid this: When collecting responses, ensure
your original respondents are participating, and use follow-up
surveys and alternates modes of reaching them if they don’t
initially respond. You can also use different channels to reach
your audience like in person, web surveys, or SMS.
5. Measurement
Measurement error is generated by the measurement
process itself, and represents the difference between the
information generated and the information wanted by the
researcher. Generally, there is always some small level of
measurement error due to uncontrollable factors.
Example: A retail store would like to assess customer
feedback from at-the-counter purchases. The survey is
developed but fails to target those who purchase in the
store. Instead, the results are skewed by customers who
bought items online.
How to avoid this: Double check all measurements for
accuracy and ensure your observers and measurement takes
are well trained and understand the parameters of the
experiment. While not all of these errors can be completely
avoidable, recognizing them is half the battle. Next time
you’re starting a research project, use this blog as a
checklist to ensure you’re doing everything you can to avoid
these common mistakes. Also, before you begin your next
research project, read 5 Ways to Formulate the Research
Problem. This is vital to any research project because you can’t
begin creating surveys unless you understand the research
problem. Once you’re ready to begin creating your survey, use
a free Qualtrics account to get started and download the eBook
below for an in-depth guide
to creating your survey questions.

Objective is clear concern and declarative sentence which

provide direction to investigate solution or variable

Characteristic of Objective the objectives of a project

should be SMART. They should be,
Specific. The problem be specifically tested.
Measurable. It is easy to measure by using research
instruments, apparatus or
equipment.
Achievable. The data are achievable using correct statistical
tools to arrive at precise
results.
Realistic. Real results are attained because they are gathered
scientifically and not
manipulated or manoeuvred.
Time-bound time frame is required in every activity because
the shorter completion
of the activity, the better.

Approaches of investigation of solutions for research problem

Research approach can be divided into three types:

1.Deductive research approach
2.Inductive research approach
3.Abductive research approach

The relevance of hypotheses to the study is the main

distinctive point between
deductive and inductive approaches. Deductive approach tests
the validity of assumptions
(or theories/hypotheses) in hand, whereas inductive approach
contributes to the emergence
of new theories and generalizations. Abductive research,
on the other hand, starts with
‘surprising facts’ or ‘puzzles’ and the research process is
devoted their explanation.
The following table illustrates the major differences
between deductive, inductive and
abductive research approaches in terms of logic,
generalizability, use of data and theory.

Deduction
Induction
Abduction
Logic
In a deductive inference,
when the premises are
true, the conclusion must
also be true
In an inductive inference,
known premises are used
to generate untested
conclusions
In an abductive inference,
known premises are used to
generate testable conclusions
Generalizability
Generalising from the
general to the specific
Generalising from the
specific to the general
Generalising from the
interactions between the specific
and the general
Use of data
Data collection is used to
evaluate propositions or
hypotheses related to an
existing theory
Data collection is used to
explore a phenomenon,
identify themes and
patterns and create a
conceptual framework
Data collection is used to explore
a phenomenon, identify themes
and patterns, locate these in a
conceptual framework and test
this through subsequent data
collection and so forth
Theory
Theory falsification or
verification
Theory generation and
building
Theory generation or
modification; incorporating
existing theory where
appropriate, to build new theory
or modify existing theory
Differences between deductive, inductive and abductive
approaches

Discussion of research approach is a vital part of any scientific

study regardless of the
research area. Within the methodology chapter of your
dissertation to you need to explain
the main differences between inductive, deductive and
abductive approaches. Also, you need
to specify the approach you have adopted for your
research by breaking down your
arguments into several points.
Deductive Research Approach
If you have formulated a set of hypotheses for your dissertation
that need to be confirmed or
rejected during the research process you would be
following a deductive approach. In
deductive approach, the effects of labour migration within
the EU are assessed by
developing hypotheses that are tested during the research
process.
Dissertations with deductive approach follow the following
path:
Inductive Research Approach
Alternatively, inductive approach does not involve formulation
of hypotheses. It starts with
research questions and aims and objectives that need to be
achieved during the research
process.
Inductive studies follow the route below:
Abductive Research Approach
In abductive approach, the research process is devoted to
explanation of ‘incomplete
observations’, ‘surprising facts’ or ‘puzzles’ specified at the
beginning of the study.

CONCEPT OF DATA COLLECTION

Data collection is the process of gathering and measuring

information on variables of interest, in an established
systematic fashion that enables one to answer stated
research questions, test hypotheses, and evaluate
outcomes. The data collection component of
research is common to all fields of study including physical and
social sciences, humanities, business, etc. While methods
vary by discipline, the emphasis on ensuring accurate and
honest collection remains the same. The goal for all data
collection is to capture quality
evidence that then translates to rich data analysis and allows
the building of a convincing
and credible answer to questions that have been posed.
Regardless of the field of study or
preference for defining data (quantitative,
qualitative), accurate data collection is
essential to maintaining the integrity of research. Both the
selection of appropriate data
collection instruments (existing, modified, or newly
developed) and clearly delineated
instructions for their correct use reduce the likelihood of errors
occurring.

Data collection is one of the most important stages in

conducting a research. You can
have the best research design in the world but if you cannot
collect the required data you
will be not be able to complete your project. Data collection is a
very demanding job which
needs thorough planning, hard work, patience,
perseverance and more to be able to
complete the task successfully. Data collection starts with
determining what kind of data
required followed by the selection of a sample from a
certain population. After that, you
need to use a certain instrument to collect the data from the
selected sample.
TYPES OF DATA

Data are organized into two broad categories: qualitative and

quantitative.

Qualitative Data: Qualitative data are mostly non-

numerical and usually descriptive or
nominal in nature. This means the data collected are in the
form of words and sentences.
Often (not always), such data captures feelings, emotions,
or subjective perceptions of
something. Qualitative approaches aim to address the
‘how’ and ‘why’ of a program and
tend to use unstructured methods of data collection to fully
explore the topic. Qualitative
questions are open-ended. Qualitative methods include focus
groups, group discussions and
interviews. Qualitative approaches are good for further
exploring the effects and unintended
consequences of a program. They are, however, expensive
and time consuming to
implement. Additionally the findings cannot be generalized
to participants outside of the
program and are only indicative of the group involved.
Qualitative data collection methods play an important
role in impact evaluation
by providing information useful to understand the
processes behind observed results
and assess changes in people’s perceptions of their well-
being. Furthermore qualitative
methods can be used to improve the quality of survey-based
quantitative evaluations by
helping generate evaluation hypothesis; strengthening the
design of survey questionnaires
and expanding or clarifying quantitative evaluation findings.
These methods are
characterized by the following attributes -
they tend to be open-ended and have less structured
protocols (i.e., researchers may
change the data collection strategy by adding, refining, or
dropping techniques or
informants);
they rely more heavily on interactive interviews;
respondents may be interviewed
several times to follow up on a particular issue, clarify concepts
or check the reliability of
data;
they use triangulation to increase the credibility of their
findings (i.e., researchers
rely on multiple data collection methods to check the
authenticity of their results);
PROCESS ANALYSIS
A step-by-step breakdown of the phases of a process, used to
convey the inputs, outputs, and
operations that take place during each phase. A process
analysis can be used to improve
understanding of how the process operates, and to determine
potential targets for process
improvement through removing waste and increasing
efficiency. Inputs may be materials,
labor, energy, and capital equipment. Outputs may be a
physical product (possibly used as
an input to another process) or a service. Processes can
have a significant impact on the
performance of a business, and process improvement can
improve a firm’s competitiveness.
The first step to improving a process is to analyze it in order to
understand the activities,
their relationships, and the values of relevant metrics. Process
analysis generally involves
the following tasks-
Define the process boundaries that mark the entry points of
the process inputs and the
exit points of the process outputs.
Construct a process flow diagram that illustrates the various
process activities and their
interrelationships.
Determine the capacity of each step in the process. Calculate
other measures of interest.
Identify the bottleneck, that is, the step having the lowest
capacity.
Evaluate further limitations in order to quantify the impact of
the bottleneck.
Use the analysis to make operating decisions and to improve
the process.
Process Analysis Tools
When you want to understand a work process or some part of a
process, these tools can help
Flowchart: A picture of the separate steps of a process
in sequential order, including
materials or services entering or leaving the process (inputs
and outputs), decisions that
must be made, people who become involved, time involved at
each step and/or process
measurements.
Failure Mode Effects Analysis (FMEA): A step-by-step approach
for identifying all possible
failures in a design, a manufacturing or assembly process,
or a product or service;
studying the consequences, or effects, of those failures;
and eliminating or reducing
failures, starting with the highest-priority ones.
Mistake-proofing: The use of any automatic device or
method that either makes it
impossible for an error to occur or makes the error
immediately obvious once it has
occurred.
Spaghetti Diagram: A spaghetti diagram is a visual
representation using a continuous flow
line tracing the path of an item or activity through a process.
The continuous flow line
enables process teams to identify redundancies in the
work flow and opportunities to
expedite process flow.
Process Flow Diagram The process boundaries are defined by
the entry and exit points of
inputs and outputs of the process. Once the boundaries
are defined, the process flow
diagram (or process flowchart ) is a valuable tool for
understanding the process using
graphic elements to represent tasks, flows, and storage. The
following is a flow diagram for a
simple process having three sequential activities-
The symbols in a process flow diagram are defined as follows-
Rectangles - represent tasks.
Arrows - represent flows. Flows include the flow of material
and the flow of information.
The flow of information may include production orders and
instructions. The information
flow may take the form of a slip of paper that follows the
material, or it may be routed
separately, possibly ahead of the material in order to ready the
equipment. Material flow
usually is represented by a solid line and information flow by a
dashed line
helps obtain population generalizability, or the degree to
which a sample represents the
population.
Content validity refers to the appropriateness of the
content of an instrument. In other
words, do the measures (questions, observation logs, etc.)
accurately assess what you want
to know? This is particularly important with achievement
tests. Consider that a test
developer wants to maximize the validity of a unit test
for 7th grade mathematics. This
would involve taking representative questions from each of
the sections of the unit and
evaluating them against the desired outcomes.

Reliability can be thought of as consistency. Does the

instrument consistently measure what
it is intended to measure? It is not possible to calculate
reliability; however, there are four
general estimators that you may encounter in reading
research:
1.Inter-Rater/Observer Reliability: The degree to which
different raters/observers give
consistent answers or estimates.
2.Test-Retest Reliability: The consistency of a measure
evaluated over time.
3.Parallel-Forms Reliability: The reliability of two tests
constructed the same way, from
the same content.
4.Internal Consistency Reliability: The consistency of results
across items, often measured
with Cronbach’s Alpha.

Relating Reliability and Validity

Reliability is directly related to the validity of the
measure. There are several important
principles. First, a test can be considered reliable, but not valid.
Consider the SAT, used as a
predictor of success in college. It is a reliable test (high scores
relate to high GPA), though
only a moderately valid indicator of success (due to the lack
of structured environment –
class attendance, parent-regulated study, and sleeping habits
– each holistically related to
success).
Second, validity is more important than reliability. Using
the above example, college
admissions may consider the SAT a reliable test, but not
necessarily a valid measure of other
quantities colleges seek, such as leadership capability,
altruism, and civic involvement. The
combination of these aspects, alongside the SAT, is a more
valid measure of the applicant’s
potential for graduation, later social involvement, and
generosity (alumni giving) toward the
alma mater.
Finally, the most useful instrument is both valid and reliable.
Proponents of the SAT argue
that it is both. It is a moderately reliable predictor of future
success and a moderately valid
measure of a student’s knowledge in Mathematics, Critical
Reading, and Writing.