Module 4: Learning Objectives

By the end of this module, you will be able to:

 Explain how reliability and validity can influence interpretation of research results.

Research is designed to gather accurate information to explain concepts or events that are not well
understood. Gathering accurate information is a critical part of research. Information gathered for
research can be collected in many ways depending on the type of research design that is being used. The
method used to gather information depends on the questions that the research will attempt to answer.

Section 1: Examples of Information Collection

Methods

There are many ways to collect information in research. The method that is chosen by the researcher
depends on the research question that is being asked. Examples of information collection methods
include surveys, interviews, tests, physiological assessments, observations, existing record reviews and
biological samples.

A survey is a set of questions for research participants to answer. Surveys can be administered in person,
through the mail, telephone or electronically (e-mail or Internet). A survey can be administered to an
individual or in a group setting. Surveys are used to gain information about many individuals and may
include multiple/forced choice or open-ended questions (e.g., demographics , health, knowledge,
opinions, beliefs, attitudes or skills).
An interview is an interaction that involves the researcher and the participant(s) in which questions are
presented in person, over the telephone or even electronically (email or Internet). During an interview,
questions are asked to obtain detailed information from the participant about the topic under study. The
questions may be similar to those asked in a survey.

A test is a form or a physical or mental task for which a normal standard has been determined or for
which there are correct answers. A participant's performance on the test is then compared to these
standards and/or correct answers. Tests are used in research to determine a participant's aptitude, skill,
knowledge, health, or mental status as compared to the general population. Tests can be administered in
person or through a paper or electronic medium. An example would be students taking a standardized
test for academic achievement (e.g., NCAE, LET).

Physiological assessments are measurements in which a participant's physical characteristics are

evaluated such as blood pressure, heart rate, or physical strength. In health-related research,
physiological assessment may be used to determine the participant's health status prior to, during, or
after the completion of the study. An example would be older adults touching their toes to assess flexibility
and reach.

Observations are recordings that are taken of the participant without requiring interaction. These
recordings are made while participants are engaged in routine behaviors and are used as an indicator of
what participants actually do rather than relying completely on self-reports of participants' behaviors. An
example would be a researcher observing an ongoing lesson plan used in a classroom by a public school
teacher.

Record reviews take place when the researcher examines and extracts information from documents that
include information about the participant. Records that are reviewed in research may be either public or
private. An example is a researcher collecting information about a disease from patient medical records.

Biological samples are substances (blood, urine, saliva) that are taken from an individual and used to
measure physiological information. An example would be drawing blood to assess the sugar content in a
diabetic patient.

No matter what kind of information is collected in a research study or how it is collected, it is extremely
important to carry out the collection of the information with precision (i.e., reliability), accuracy (i.e.,
validity), and minimal error. The integrity and usefulness of the research may be compromised if the study
measurements are not carried out correctly. Factors that contribute to effective information gathering are
discussed in the next few sections.
Section 2: Factors in Effective Information Gathering

Three major concepts are

important to understand in order to collect useful and valuable research information: precision (i.e.,
reliability), accuracy (i.e., validity), and error.

2.1 Precision (i.e., Reliability)

It is very important to make sure the information gathering methods are precise (i.e., reliable). This means
that a method measures the same thing every time you use the method.

There are many things that can affect an instrument's

or a method's precision (i.e., reliability). This includes the form of the instrument (e.g. oral or written), the
environment in which it is administered, how the staff administers it, the difference in participants from
group to group and across time, and the time of the day it is administered (among many others). The
researcher can also influence precision (i.e., reliability) by either overtly or inadvertently praising,
complimenting or admonishing the participant. The principal investigator is responsible for providing
proper training and “spot checks” for how instruments or methods are administered to ensure that the
research study is conducted with precision.
Research studies are often criticized because they did not use precise methods to gather data. Precision
(i.e., reliability) helps to promote research that is of greater value because you can be more confident that
the findings are real.

2.2 Accuracy (i.e., Validity)

It is also very important to make sure the information

gathering methods are accurate (i.e., valid). Accuracy (i.e., validity) refers to whether or not an instrument
or method truly measures what you think it measures. Researchers want accurate or valid study
procedures so that study results are useful and meaningful.

There are many things that affect the accuracy (i.e., validity) of an instrument or method. These include
the cultural appropriateness, the theoretical constructs used to develop the instrument or method, and the
appropriateness of the testing method or form of the instrument for the participants' abilities (among many
others).

 
 To demonstrate that study
measures are accurate, researchers will sometimes collect different kinds of information to measure the
same thing. They then look to see if all of the methods or instruments provide the same or similar
conclusions. If they do, the researcher can be confident that what they have found actually represents
what they intended to study.

Along with lack of precision (i.e., reliability), research studies are often criticized because of the use of
inaccurate methods to gather information. Measurement accuracy (i.e., validity) is essential in order to
guarantee the quality and integrity of the research findings.

Definition: Accuracy (i.e., validity) refers to whether or not an instrument or method truly measures what
you think it measures.

Case Example for Accuracy (i.e., Validity)

In a study involving a weight loss program, the researcher weighs participants to determine if the program
is effective in helping individuals lose weight. To accurately measure weight, the scale must be working
properly. To check the accuracy of the scale, a 25 pound weight is placed on a digital scale three times to
make sure that a 25 pound reading is found each time.
In another study, researchers want to determine whether participants have reduced their smoking. To find
out, the researcher asks the individual a series of questions on a survey about their smoking habits over
the last two weeks. To verify the accuracy of the survey responses, the researcher also does a test to
measure chemicals in saliva that increase with smoking.

Section 2.2: Discussion Questions

1. What would happen in each of the previous examples if an accuracy check was not conducted?

2. What are some other ways to check for accuracy in these studies?
3. a. give the results in your group research
b. How did you check for accuracy in your group research?
c.was there an instance that accuracy check was not conducted in your data?9

2.3 Error

When we measure something or collect information,

there are many reasons that our findings might be wrong. The most obvious reason is that we could
simply make a mistake in writing something down. This kind of mistake is how we might usually think
about error. However, there are other kinds of errors that we might not see unless we knew to look for
them. These errors are not mistakes in the sense that we have done something wrong. These types of
errors can decrease the reliability or accuracy of what we do, but often because of things that we cannot
control.

then we measure something or collect information, there are many reasons that our findings might be
wrong. The most obvious reason is that we could simply make a mistake in writing something down. This
kind of mistake is how we might usually think about error. However, there are other kinds of errors that we
might not see unless we knew to look for them. These errors are not mistakes in the sense that we have
done something wrong. These types of errors can decrease the reliability or accuracy of what we do, but
often because of things that we cannot control.
 

Module 4: Quiz Questions

1. At what point in the study is it most appropriate to document information:

1. Months after the study is completed.

2. After you’ve had time to think about your personal interpretation of what you observed.
3. After you talk to all of your colleagues about the most accepted way of documenting the
results.
4. None of the above

. 2. A study involves a self-report survey completed by the participant. Will the results be
affected if the research assistant reads the questions to the person and records the
answers?

a. a.Yes, this change in how the survey is administered can change the results of the
study.
b.No, how the survey is administered doesn’t make a difference.

. 3. It is important for measurements in research to be accurate or valid so that:

a. a.The method you are using measures what you think you are measuring.
b.The method you are using measures the same thing every time the method is used.
c.You are confident that the method you are using is precise.
d.The method you are using finds the answer you expected to find.

. 4. It is important for measurements in research to be precise or reliable so that:

a. a.The method you are using measures the same thing every time the method is used.
b.The method you are using measures what you think you are measuring.
c.You are confident that the method you are using is accurate.
d.The method you are using finds the answer you expected to find

. 5. In your research, what is acceptable and consistent to use for one respondent to
another to attain reliable results?
.
. (Pls attach the sample instrument used)

Module 5: Learning Objectives

 Explain the importance of good data in ensuring research integrity.
 Describe why changing or manipulating data can compromise interpretation of the results.

To maintain the confidentiality and accuracy (i.e., validity) of the information collected during research,
any information collected about an individual should be handled with care through proper documentation
and secure storage. Failure to properly handle research information may result in wasting resources,
violating confidentiality, and failing to answer the research question(s) (i.e., adding unknown (random)
error to the study). To avoid these serious consequences, research information should be properly
documented and stored and the information provided by a research participant should never be changed
or altered.

Section 1: Security and Storage

The ethical conduct of research, with

human subjects especially, depends in part on protecting the confidentiality of sensitive information such
as name, age, occupation, income level, education level, job title, health status, family history,
photographic images, survey answers, and interview responses. Even biological samples may reveal
information about an individual that could be very sensitive (such as a positive HIV test). Carefully
handling information according to the research design will help to ensure that sensitive information
collected for research will remain private.

Research information should always be stored in a secure location. Forms and other study materials that
include confidential information (e.g., blood or tissue samples, audio/video-tapes, informed consent
forms) should be stored in a locked facility to which only authorized personnel have a key. Computers
should be password protected and the user should log-off when the computer is not in use.

1.1 Documentation
The ethical conduct of research is also measured by the degree to which the information collected is
accurate and precise. Information should be recorded and documented carefully according to the
research design so that the potential for error is reduced. Documentation is the act of recording,
summarizing and/or coding information for future use.

Documentation must be accurate so that the records of research activity represent what
actually occurred during testing/participation. Accurate documentation also allows others to more easily
detect any errors that may be included in the information collected. If information is recorded inaccurately,
the results of the study may not be useful.
Documentation should be completed as the research occurs, not at a later time. Information is likely to be
more accurate when recorded as soon as possible rather than recalling information from memory, which
can be mistaken.

Definition: Documentation is “the collation, synopsizing, and coding of printed material for future
reference” (American Heritage Dictionary of the English Languages).

Case Example for Documentation

 In a weight loss study, the research assistant failed to measure the participant's weight at the beginning
of the study. Later, when it was time to review the results of the study, the assistant realized that he did
not have the weight for the participant.

Section 1.1: Discussion Questions

1. How does this type of situation affect the integrity of the study?

2. What should you do if you forget to document important information in the beginning of the study?

3. Why is it important to accurately document research information?

Module 5: Quiz Questions

1. Which of the following is not an appropriate way to store confidential research
information?

a.Receiving data on a password-protected computer to which research staff only have access.
b.Storing completed questionnaires in a locked filing cabinet.
c.Assigning codes to identifiable information and storing the master code list in a separate location from
the data.
d.Leaving completed surveys that include sensitive information with the participant's name and contact
information on the desk of the research staff member.

a. Why is research information often stored for several years?

a.Because it is illegal to destroy research information.
b.Because it may be necessary to prove the authenticity of the information.
c.So that the researcher can make photocopies to distribute to the other participants.
d.For liability purposes in the case of a law-suit.

.
.
 .
.
.
.
.
.

. 3. In a study of the “over 55” population, a researcher recruits a 54-year-old woman, but
notes in the research records that she is 56 years old. How does this affect the study
results?

a.This decision will help senior citizens over the age of 55.
b.This decision will cause misleading and invalid study results.
c.This decision will not affect the study results in any way.
d.This decision will help the 54 year-old woman in the study

. 4.When is it appropriate to change research information?

a.When the participants provide controversial responses
b.When it appears that the results will disprove the researcher's hypothesis
c.When there is a missing response in the research record
d.It is never acceptable to change research information

. 5. In a study of the “over 55” population, a researcher recruits a 54-year-old woman, but
notes in the research records that she is 56 years old. How does this affect the study
results?

a.This decision will help senior citizens over the age of 55.
b.This decision will cause misleading and invalid study results.
c.This decision will not affect the study results in any way.
d.This decision will help the 54 year-old woman in the study

ANSWERS
Module 4: Quiz Questions
1. At what point in the study is it most appropriate to document information:

1. Months after the study is completed.

2. After you’ve had time to think about your personal interpretation of what you
observed.
3. After you talk to all of your colleagues about the most accepted way of
documenting the results.
4. None of the above
 . A study involves a self-report survey completed by the participant. Will the results
be affected if the research assistant reads the questions to the person and records
the answers?

a. A. Yes, this change in how the survey is administered can change the results of
the study.
b. No, how the survey is administered doesn’t make a difference.

. It is important for measurements in research to be accurate or valid so that:

a. The method you are using measures what you think you are measuring.
b. The method you are using measures the same thing every time the method is used.
c. You are confident that the method you are using is precise.
d. The method you are using finds the answer you expected to find.
d.The method you are using measures what you think you are measuring.

. It is important for measurements in research to be precise or reliable so that:

a. The method you are using measures the same thing every time the method is used.
b. The method you are using measures what you think you are measuring.
c. You are confident that the method you are using is accurate.
d. The method you are using finds the answer you expected to find
d.The method you are using measures the same thing every time the method is used.

. In a study involving blood pressure, it is acceptable and consistent to use a manual cuff
for one participant and a digital blood pressure machine for another participant.

a. A. True, the type of blood pressure machine would not affect the results.
b. False, the type of blood pressure machine that is used should be consistent, or
the same, for each participant.

Module 5: Quiz Questions

2. Which of the following is not an appropriate way to store confidential research
information?

a.Receiving data on a password-protected computer to which research staff only have access.
b.Storing completed questionnaires in a locked filing cabinet.
c.Assigning codes to identifiable information and storing the master code list in a separate location from
the data.
d.Leaving completed surveys that include sensitive information with the participant's name and contact
information on the desk of the research staff member.

b. Why is research information often stored for several years?

a.Because it is illegal to destroy research information.
b.Because it may be necessary to prove the authenticity of the information.
 c.So that the researcher can make photocopies to distribute to the other participants.
d.For liability purposes in the case of a law-suit.

.
.
.
.
.
.
.
.

. 3. In a study of the “over 55” population, a researcher recruits a 54-year-old woman, but
notes in the research records that she is 56 years old. How does this affect the study
results?

a.This decision will help senior citizens over the age of 55.
b.This decision will cause misleading and invalid study results.
c.This decision will not affect the study results in any way.
d.This decision will help the 54 year-old woman in the study

. 4.When is it appropriate to change research information?

a.When the participants provide controversial responses
b.When it appears that the results will disprove the researcher's hypothesis
c.When there is a missing response in the research record
d.It is never acceptable to change research information

. 5. In a study of the “over 55” population, a researcher recruits a 54-year-old woman, but
notes in the research records that she is 56 years old. How does this affect the study
results?

a.This decision will help senior citizens over the age of 55.
b.This decision will cause misleading and invalid study results.
c.This decision will not affect the study results in any way.
d.This decision will help the 54 year-old woman in the study