Chapter 10

DATA AT SCALE AND ETHICAL CONCERNS

 Goals
• Provide an overview of some potential
impacts of data at scale.
• Introduce key methods for collecting
data at scale.
• Discuss how data at scale is used in
interaction design.
• Review the key methods for visualizing
and exploring data at scale.
• Introduce privacy and other ethical
design concerns with data at scale and
AI.
www.id-book.com
 What is data at scale?
• Data at scale is also known as Big Data.
• It describes numbers, images of people,
places and things, conversations, video
recordings, sensor data, etc.
• Huge potential for solving problems.
• Can be beneficial to individuals and society.
• But need to know how to collect, analyze,
and communicate findings.
• Can involve dangers to people’s privacy.
www.id-book.com
 Collecting data at scale?
• Easy to collect masses of data, but then what do
you do with it?
• What are the rules and policies that apply when
collecting data about people?
• For example, tracking people’s activities as they
move through Heathrow and other airports for
security reasons may also violate their privacy.
• Also, scraping and crowdsourcing data, personal
data collection directly and indirectly through
devices, and sentiment and social network analysis.
www.id-book.com
Collecting personal data at
Heathrow airport

Source: Mark Zakian / Alamy Stock Photo

www.id-book.com
 Asking questions of data
• Google Trends can be used for exploring the
motivation behind what people ask in Google
Search
– https://trends.google.com
– try typing in “I feel happy’ and then ‘I feel sad’
• To obtain new insights from big data requires
asking the right questions (Stephens-Davidowitz,
2018)
• For example, what questions should HCI
researchers ask to find out how people relate to,
trust, and confide in voice technologies, such as
smart speakers?

www.id-book.com
Crowdsourcing volunteered data:
Smartphone Covid-19 app

Survey to collect symptoms Recording voice samples

Source: Cecilia Mascollo
www.id-book.com
Citizen science and privacy

Source: iNaturalist.org
www.id-book.com
 Sentiment analysis
• A technique to infer what a group of people or crowd
is feeling or saying.
• Posts on social media are scored: negative, positive,
or neutral.
• Scales can vary, e.g, +10 most positive, -10 most
negative, 0 neutral.
• Scores can also be associated with specific feelings,
e.g. anger, sadness or fear are negative, happiness,
joy, enthusiasm are positive.
• Data is analyzed using algorithms.

www.id-book.com
 Social network analysis (SNA)
• A social network is a network of people and
groups with relationships to each other
• At an individual level, SNA is about “who you
know”
• At the group level, it shows how each
person’s individual connections aggregate to
form connected sub-groups
• This can be visualized using maps to show
better the conbections and patterns
www.id-book.com
Social network analysis

Source: Derek Hansen and Mark Smith, 2014.

www.id-book.com
 How do you probe people more?
• Askig people what they think about a tracking
technology may not reveal the true extent of their
concerns and feelings.
• What other methods could be used?

• One approach is to use a provocative probe which

the Quantified Toilets project did
– set up a fake service in a public place to disrupt the
accepted norms
– all sampled data from the toilets were anonymized but
also fake so not belonging to anyone

www.id-book.com
 Probing people’s reactions to
personal tracking

Source: Courtesy of Quantified Toilets www.id-book.com

Combining multiple sources of data
• Many projects combine automatic sensing
and subjective reporting
– to obtain a more comprehensive picture about
a domain
• But how to do this and what questions to
ask of the masses of data?
• Often concerned with trends over time and
events that impact
– e.g. students’ mental health during a semester

www.id-book.com
Students’ activities

StudentLife Study

Source: StudentLife Study

www.id-book.com
 What do the two graphs say about the students' activity,
their stress levels, and their level of socializing in relation
to deadlines over the course of the term?

StudentLife Study

Source: StudentLife Study

www.id-book.com
 Visualization of big data
• Goal of data visualization tools is to amplify
human cognition
– enable people to see patterns, trends,
correlations, and anomalies
– Lead to new insights and new discoveries
• People can can zoom in and out of the
visualisation to see an overview or get details
• Ben Shneiderman (1996) summarizes this
behavior in his mantra “overview first, zoom
and filter, and then details on demand.”
www.id-book.com
An example of a data visualization
tool: Market map of the S&P 500

Source: Used courtesy of FINVIZ

www.id-book.com
A Community-Based Environmental
Data Toolkit: Smart Citizen

Source: CitizenMe, www.citizenme.com

www.id-book.com
 Visualization of different sounds
including birds and insects

ACM Publications

Source: Used courtesy of Oliver et al., 2018. Permission from ACM Publications.

www.id-book.com
Dashboard showing changes
in sales information

Zoho Analytics

Source: Zoho Analytics

www.id-book.com
 Exemplar dashboards

Source: Sankaya et al., 2018. Reproduced with permission of IEEE

www.id-book.com
Weather data

Actual weather
data

Wundermap of
data from the
same area at
the same time
Source: Weather Underground
www.id-book.com
 Two different visualizations
of bat call data
(a) General
visualization
for the public

(b) Spectrogram for

scientists

Source: (a) Used courtesy of Matej Kaninsky; (b) Used courtesy of Sarah Gallacher
www.id-book.com
 New kinds of visualization tools
• A challenge is how to make powerful tools available to
more people who want to explore a diversity of data
• AI techniques are now incorporated in the tools that
automate many data analytic tasks
– makes it easier for business people to use
• Natural language interfaces have also been developed
making it easier for people to ask specific questions,
e.g.
– Tableau’s Ask Data lets someone type a question in
everyday language such as “show the total sales for the
first quarter”
• Telling stories can also help illustrate the context that
the data visualizations represents

www.id-book.com
 Ethical design concerns (1)

• As more data is collected, displayed and

acted upon people are becoming more aware
of privacy issues, e.g.
– Companies can provide feedback on energy
consumption in a building to encourage workers
to reduce their energy consumption.
– However, what if localized occupancy rates or
energy consumption for each room were shown?
– It would not take much to figure out who was in
that space.
– Would that be a step too far and an invasion of
their privacy? Would people mind?

www.id-book.com
Graphical display of average daily
energy consumption for a building

Source: Yvonne Rogers

www.id-book.com
 Ethical Design Concerns (2)

• How can HCI be involved in designing and

evaluating systems that use big data,
through conducting research that follows
human-centered codes of practice?
• How to ensure AI systems from making
unseemly mistakes and amoral decisions.
• Have come up with core design principles
– fair, honest, trustworthy, secure and respectful
of privacy

www.id-book.com
 Data ethics principles
Fairness: Is the treatment just and without
favoritism or discrimination?
Accountability: Is the data accurate and
correct?
Transparency: Are the decisions being made by
a system visible?
Explainability: Can people who are not experts
understand the explanations provided by the
system?
Privacy: Is personal information and data that is
collected about and from speople socially
acceptable? www.id-book.com
 Salency map used in explainable AI

Map on the right is an explanation of how the dog

image on the left was classified by a deep learning
algorithm
Source: CNN, https://usmanr149.github.io/urmlblog/cnn/2020/05/01/Salincy Maps.html
www.id-book.com
DeepCam’s face-tracking software

Source: DeepCam
www.id-book.com
 Summary
• Data at scale involves very large volumes of data, also
known as Big Data.
• The data can be qualitative or qualitative.
• The data comes from different sources, eg. sensors,
social media, documents, facial recognition, audio,
video surveillance.
• Data can be collected by data scraping, monitoring
oneself, crowdsourcing, sentiment analysis, social
network analysis.
• Analyzing data from different sources can provide
different perspectives to answer questions.
• Data visualization and other tools help to show the
trends in data at scale.
www.id-book.com
 Summary (continued)
• Data analysis can have both positive and negative
impacts on individuals and society.
• Positive impacts include understanding more about
health; negative impacts include revealing people’s
personal details.
• Privacy is becoming a increasing concern because data
from different sources can be brought together to
identify individuals and reveal patterns of behavior.

www.id-book.com