Assignment: Operationalization and data

collection
This assignment aims to teach you the differences between different types of data
collection and how to conceptualize and operationalize concepts.
Read this assignment carefully and answer all questions.
Hand in the assignment via Canvas.

Part A. Conceptualization, operationalization, and measurement: terminology

1. Write down, in your own words, what is meant by “conceptualization”, “operationalization” and
“measurement”.
For example:
Conceptualization: To make clear what is meant by a certain theoretical variable (construct), its
aspects/dimensions and its attributes/values.
Operationalization: To specify the exact procedure of how to measure the concept.
Measurement: To actually measure the concept (using a certain data collection method). Or, in other
words, following a specified procedure (data collection method) to actually get the observations or
data.
2. Look at the following definitions. Is the definition a conceptualization or an operationalization of
the construct?
Definitions are conceptual when they are describing what is meant with the concept, but it is still not
clear how you are going to measure the concept when reading it.

Definition Conceptualization Operationalization

Mental health is a state of emotional, psychological and
X
social well-being.
Stress is determined by a self-assessment quiz of 10
X
items.
Self-esteem is the subjective positive / negative
X
evaluation of the self.
Attitude toward money has more to do with the
psychological value of money than its relative economic X
value.
Motivation can be divided into intrinsic and extrinsic
X
motivation.
The OECD defines poverty as the ratio of the number of
people (in a given age group) whose income falls below X
the poverty line; taken as half the median household
income of the total population.
Highly gifted pupils are pupils with an IQ of at least 130. X

1
 Trust is considered as a firm belief in the character,
X
strength, or truth of someone or something

Part B. Conceptualizing and operationalizing of feminism

3. Let’s examine the concept of feminism. Conceptualize the word feminism.

We can conceptualize ‘feminism’ as a variable characterizing a person (the unit). This is a choice: we
can also conceptualize ‘feminism’ as a characteristic of a book, a political party or a country. Always,
first make clear which type of unit ‘feminism’ is characterizing.

A conceptualization of ‘feminism’ with which we can characterize people is: ‘The extent to which a
person actually supports women's rights on the grounds of political, social, and economic equality to
men’. Or you could conceptualize it as the belief that men and women should have equal rights and
opportunities.

Remember that your conceptualization has consequences for the choices you make when you
operationalize the concept. If you conceptualize feminism as ‘support’ you would probably look for
opinions or behaviors or both, while if it is conceptualized as a ‘belief’ you would probably look for
opinions only.

4. Write down at least two very different methods of data collection you could use to measure
feminism of a person (as conceptualized by you).

a. Survey with questions about attitudes

b. Observing behavior (possibly induced in a specified setting)
c. Analyzing texts written by the people you want to observe (content analysis)

5. Suppose you want to measure the amount of feminism in a population of people, which of the
aforementioned methods would you prefer? Why? Give some pros and cons of the methods to
support your answer.

Observing is very time-consuming and difficult to plan. Content analysis of texts is only possible if
there are text sources that are easy to find and access. Analysis of texts by hand is very time
consuming and automatic text analysis is not (yet) suited to complex questions. If we want to access
opinions of large groups of people, a survey (online) can be very efficient, so this is probably the best
way to go forward.
Note: when you make a choice of data collection method you typically also take into account what
access you have to the population (see unit 19).

6. Researchers have used the Eurobarometer (check on the internet to see what the Eurobarometer
is) to measure feminist attitudes. Respondents were asked to answer five questions on a 4-point
scale (scored here from zero to three) ranging from “completely disagree” to “completely agree”):

(1) fighting against people who would like to keep women in a subordinate role,
(2) obtaining “true equality between women and men in their work and careers,”
(3) political parties giving women equal opportunities to be party leaders or candidates,
(4) achieving gender equality in responsibilities for childcare, and
2
(5) organizing “women into an independent movement to achieve a radical transformation of
society.

In addition, respondents were asked whether they had an unfavorable or favorable opinion of the
women’s movement (also scored from zero to three).1

Is using these data from the Eurobarometer by researchers not connected to the project an example
of primary data analysis or of secondary data analysis?

Secondary, data is collected by others, but other researchers can download and analyze them.

7. What would be relevant reasons to either use primary or secondary data?

Secondary data are available (pro) but designed and formulated on the basis of a conceptualization
which may be different from yours (and thus may not be valid) (con).

1
Banaszak, L., &Plutzer, E. (1993). The Social Bases of Feminism in the European Community. The Public
Opinion Quarterly, 57(1), 29-53. Retrieved from http://www.jstor.org/stable/2749436, page 35-36.
3
Part C. Data and data collection

8. When researchers decide which data to use, there are several things to consider. First, will they use primary or secondary data. Second, if they use primary
data, do they use naturally occurring data or elicited data? Fill out the following overview with examples of data collection methods.

Elicited data How will you elicit data?

Will you elicit responses or

Primary data will you collect naturally
occurring data?
Will you collect new data
or use existing research
data?

Secondary data

Naturally occuring data How will you collect data?

4
Examples of correct answers.

5
 Survey

Interview

Focus group

Observation of behavior in
Elicited data How will you elicit data? a controlled environment
or after a prompt

Elicit spoken or written

texts in a controlled
environment or after a
prompt

Collect data using

technology, e.g. using eeg,
eyetracking etc.

Will you elicit responses or

Primary data will you collect naturally
Automatically elecit
occurring data?
reactions from software
Will you collect new data
agents
or use existing research
data?

Secondary data
Observation of naturally
occuring behavior

Select existing data, e.g.

texts, videos, patient files,
educational records etc for
traditional analyses.
Naturally occuring data How will you collect data?
Automatically select
existing data from the web
or large data archives for
analysis using data science
techniques (e.g. machine
learning)
Collect data using
6 technology, e.g. place a
hidden camera, record the
number of times a door is
opened etc.
9. In the following table, different situations in which data are collected are described. How was the data collected? Was this naturally occurring data or elicited
data? What was the goal of the study? Can you identify elements of the research question? Fill out the table for each case. The first one is included as an
example.
Situation Data collection Naturally occurring data or Exploratory, descriptive, Units, variables, setting in
method elicited data explanatory (see pages 90- the research questions
92, Babbie)
A master student reads minutes of Collecting text data Naturally occurring data Descriptive Units: topics
primary school board meetings to
Variables: put on the agenda
study the decision making process.
by teachers: yes/no
The research question is: Which
Setting: in the previous 5
topics have been put on the
years
agenda by teachers (versus school
leaders and others) in the previous
5 years?
A call center employee calls all Survey Elicited data Descriptive – The goal is to Units: patients of this
patients from a particular hospital describe the rating of the hospital who visited as a
from the last 6 months and asks hospital by its patients. patient in the past 6 months
how they rate their last hospital
Variables: ratings
visit.
Setting: The setting is limited
The research question is: What is
to one hospital
the average rating given by
hospital patients.
A researcher joins meetings of the Observation Naturally occurring data Descriptive – The researcher Units: proposals
faculty board and keeps a log of is interested in the effect of
Variables: amount of support
reactions to proposals made by the who makes the proposal on
and proposed by (student or
board members. the support of the proposal.
staff)
The research question is: Do
Setting: meetings of the
proposals from students or
faculty board
proposals from staff members

7
have a higher chance of gaining
support?
A researcher records an interview Interview Elicited data Exploratory – The key Unit: the politician
with a politician. variables are still very
Variables: motivating factors
unclear and the goal of the
The research question is: What
study is to identify them. Setting: life of the politician
were the factors (life experiences,
opinions etc) that motivated this
politician to go into politics?
A video streaming service invites Focus group Elicited data Exploratory – see previous Units: elements of the
10 customers to their office and streaming service
holds a group discussion with
Variables: evaluations
these customers to evaluate the
streaming interface. Agreement on the
evaluations
The research question is: How do
customers evaluate the elements Setting: no setting
of the streaming service and is mentioned
there agreement on these
evaluations?
A researcher interviews board Interview Elicited data Explanatory: The research Units: decisions
members of a study association question is about the why of Variables: the proposal that
and asks them about their decisions. the decision is about
motivations to support or oppose The board member who took
5 recent proposals. the decision
The research question is: Do Whether personal relations
personal relations between board played a role (yes/no)
members play a role in the Setting: the board of a study
decision to support or oppose association
proposals?
A researcher uses all English Automatically collected Naturally occurring data Descriptive - The goal is to Units: tweets
language tweets from New Years data (big data) describe the tone of tweets.

8
 day to find out if people were Variables: tone (optimistic,
generally optimistic or pessimistic pessimistic, neutral, mixed)
about the new year.
The research question is: Which
proportions of tweets about the
start of 2023 were optimistic,
pessimistic, mixed, or neutral in
tone?
Two researchers invite 40 people A multiple choice test Elicited data Explanatory – the goal is to Units: participants
to their laboratory and randomly test the effect of context on
Variables: context of the
divide them into two groups. The recall
advertisement (game versus
first group sees an advertisement
cartoon)
in the context of a game. The
second sees an advertisement in Recall/ memory/ number of
the context of an entertaining questions answered correctly
cartoon. After a short break, the in the multiple choice test
participants are asked to answer
multiple choice questions about Setting: unclear
the advertisement.
The research question is: Does the
context in which an advertisement
is seen have an effect on how well
people remember it?

>>End of the assignment<<