To appear in IEEE Transactions on Visualization and Computer Graphics

Data Changes Everything:

Challenges and Opportunities in Data Visualization Design Handoff
Jagoda Walny, Christian Frisson, Mieka West, Doris Kosminsky,
Søren Knudsen, Sheelagh Carpendale, Wesley Willett

Abstract— Complex data visualization design projects often entail collaboration between people with different visualization-related
skills. For example, many teams include both designers who create new visualization designs and developers who implement the
resulting visualization software. We identify gaps between data characterization tools, visualization design tools, and development
platforms that pose challenges for designer-developer teams working to create new data visualizations. While it is common for
commercial interaction design tools to support collaboration between designers and developers, creating data visualizations poses
several unique challenges that are not supported by current tools. In particular, visualization designers must characterize and build an
understanding of the underlying data, then specify layouts, data encodings, and other data-driven parameters that will be robust across
arXiv:1908.00192v1 [cs.HC] 1 Aug 2019

many different data values. In larger teams, designers must also clearly communicate these mappings and their dependencies to
developers, clients, and other collaborators. We report observations and reflections from five large multidisciplinary visualization design
projects and highlight six data-specific visualization challenges for design specification and handoff. These challenges include adapting
to changing data, anticipating edge cases in data, understanding technical challenges, articulating data-dependent interactions,
communicating data mappings, and preserving the integrity of data mappings across iterations. Based on these observations, we
identify opportunities for future tools for prototyping, testing, and communicating data-driven designs, which might contribute to more
successful and collaborative data visualization design.
Index Terms—Information visualization, design handoff, data mapping, design process

1 I NTRODUCTION
Creating custom visualizations is a challenging, multifaceted problem environments like Observable [47]. We call this design-as-development.
that requires a combination of skills and tools for data analysis, design, However, using these low-level tools requires considerable technical
and development. Designers and developers must gain an understanding skill and can increase the time and effort needed to articulate, refine,
of the dataset and its characteristics through data exploration, then and polish visualization designs. This scales poorly for large visualiza-
design data mappings, aesthetics, and interactions based on it [6]. These tion projects, which may involve not just developers but also interaction
designs also need to be realized and deployed, typically by writing designers, data experts, and clients, each with their own tool sets and
software. Sometimes it is possible for one person to perform all of institutional processes. In these collaborative settings, differing design
these activities given enough time and resources. However, for more objectives and gaps in the tools used to characterize data, design visu-
complex visualization projects with limited timelines, it is more feasible alizations, and develop applications can exacerbate the issues caused
to distribute these activities amongst people in specialized roles. by a lack of data-driven support. While unexpected issues can be han-
This distribution of roles creates the challenge of handoff, the codify- dled with some agility by small, flexible teams, these issues are easily
ing and exchange of information between people working on different amplified in larger teams.
roles in a project, and the related challenge of communicating domain
knowledge across roles. This problem is already well-known in general
software design, where interaction designers are often distinct from
software developers [39]. Over the past two decades, a wide range of Interaction Design Visualization Design
specialized tools has emerged to help interaction designers outline and
prototype interfaces in ways that reduce the friction between graphical
designs and code. Commercial tools like Adobe XD [2], InVision [27],
and Sketch [9] support expressive and precise visual design, interactive Individual DESIGN AS DEVELOPMENT
(Developer Tools)
UNDERSTANDING & DESIGN AS DEVELOPMENT
(Developer Tools)
prototyping of animations, transitions, interactions, and streamlined Designer
or or
exporting of specifications and assets for collaborators. and/or
Unfortunately for visualization designers, these tools lack robust Developer
support for data-driven designs. In practice, many programming-literate
DESIGN DEVELOP UNDERSTAND DESIGN DEVELOP
visualization designers still work largely in code, exploring datasets
through iterative prototyping using libraries like D3 [10] or notebook (Sketch, Adobe XD, etc.) (Gap) (Data Design Tools)

• All authors are with the University of Calgary, Calgary, Alberta, Canada. Multiple
• Christian Frisson’s secondary affiliation is with McGill University, Designers
Montreal, Quebec, Canada. E-mail: [email protected]. and/or DESIGN DEVELOP UNDERSTAND DESIGN DEVELOP
• Doris Kosminsky’s primary affiliation is with Federal University of Rio de
Janeiro, Rio de Janeiro, Brazil. E-mail: [email protected].
Developers (Handoff Tools) (Gap) (Gap)

• Søren Knudsen’s secondary affiliation is with University of Copenhagen,

Copenhagen, Denmark. E-mail: [email protected].
• Sheelagh Carpendale’s primary affiliation is with Simon Fraser University,
Fig. 1: Contemporary interaction design tools increasingly enable
Vancouver, British Columbia, Canada. E-mail: [email protected]. smooth transitions and collaboration between design and development
• Remaining author e-mails: {jkwalny, mieka.west, (top-left) and handoffs between designers and developers (bottom-left).
wesley.willett}@ucalgary.ca. In our experiences across projects, these transitions remain challenging
for visualization designers (top-right) and teams (bottom-right).

1
 IDB Energy Visualization Energy Futures Pipeline Incidents Energy Imports & Exports Pipeline Conditions

Data Provider Data Provider Data Provider Data Provider Data Provider
CONCEPTUALIZATION Inter-American CONCEPTUALIZATION National Energy CONCEPTUALIZATION National Energy CONCEPTUALIZATION National Energy CONCEPTUALIZATION National Energy
Development Bank Board of Canada Board of Canada Board of Canada Board of Canada

CHARACTERIZATION CHARACTERIZATION CHARACTERIZATION CHARACTERIZATION CHARACTERIZATION

Design Team Design Team Design Team Design Team Design Team
4 Members 12 Members 13 Members 10 Members 11 Members
DESIGN *1 author
DESIGN *4 authors DESIGN *6 authors DESIGN *6 authors DESIGN *6 authors

Dev. Team Dev. Team Dev. Team Dev. Team Dev. Team
DEVELOPMENT 2 Members DEVELOPMENT DEVELOPMENT DEVELOPMENT DEVELOPMENT
5 Members 5 Members 5 Members 9 Members

Fig. 2: The five visualization design and development projects in which we ground our observations.

We draw on our experience as part of the design team on five complex by difficulties in implementing the final design. Leiva et al. [35] expand
data visualization projects (Figure 2) intended for wide-scale public on this concept, identifying several specific types of breakdowns —
release. Across each of these projects, we observed and participated including omitting critical details, ignoring edge cases, and disregard-
in interactions between teams of designers and developers working ing technical limitations — that routinely contribute to difficulties in
together and separately to characterize data, create initial designs, and projects involving handoffs between designers and developers.
translate those designs into production-ready applications. Based on However, research in data visualization design still often fails to
these experiences, we highlight challenges and opportunities specific to acknowledge this division of design and development. For example,
designer-developer collaboration in data visualization design projects. McKenna et al.’s Design Activity Framework [41] combines design
While others before us have discussed practical visualization design and implementation into a single “make” step and assumes that the
projects [31, 60] and design studies [55], our focus is different. We responsibility for both will be tightly integrated. Other reflections on
focus on the practical work and coordination that goes into building visualization design practice also tend to share this assumption, draw-
visualizations. Although our projects did employ several researchers, ing primarily on the perspectives of visualization design researchers
they also employed design and development practitioners. Our projects tasked with both creating and implementing novel visualizations for
also involved close coordination with project coordinators and data multidisciplinary collaborators [31, 55, 60].
experts1 on the data provider’s side. This complex structure and the The design and development of new visualizations, like that of other
physical and temporal separation of different teams heightened the visi- interactive systems, entails considerable iteration and involves transi-
bility of several practical challenges still faced during the visualization tions between multiple sets of tools as designs move from conception
design process. Our work articulates issues that are of a very practical to implementation. While some amount of visualization design and
nature and that we expect are frequently experienced by others engaged development often happens via coded prototypes, many aspects of visu-
in practical visualization work. We think our contributions add value alization design — from early-stage concept generation and sketching
to this practitioner-oriented research space, especially in light of the through to late stage aesthetic refinement — are often better-served by
visualization research community’s recent focus on practitioners as a graphic design and interaction design tools that offer greater expressive
crucial source of “energy, ideas, and application problems” [23]. flexibility, as evidenced by the work practices of data visualization de-
signers interviewed by Bigelow et al. [6]. However, unlike most other
2 R OLES IN V ISUALIZATION D ESIGN P ROCESSES work in interaction design, the form of new visualizations depends
Visualization design and development requires several unique sets of intrinsically on the data that they will communicate. As a result, the
skills including experience in human-centered design, perception, eval- process of visualization design is often a complex and iterative one
uation, statistics, and graphics programming [31]. This conventional anchored in multiple rounds of data examination, ideation, creation,
wisdom is exemplified by the (somewhat mythical) notion of “full-stack” and deployment [41]. These activities pose challenges for designers
visualization designer-developers capable of conducting the full range who may need to transition repeatedly between interactive tools that
of “data wrangling, dynamic graphics, and derring-do” [22]. However, allow them to examine data and explore a diverse range of designs and
real-world visualization design projects (especially large ones) often more low-level data-driven development and coding.
include a variety of team members with diverse and overlapping subsets These issues are compounded as projects grow and responsibilities
of these skills. As projects grow, these teams can become segmented, for design, development, and deployment are divided across multiple
with responsibility for design and development delegated to individu- individuals or teams, each with different skill sets and priorities. In
als or teams whose skill sets and preferred tools can be increasingly these situations, visualization design and development become an ex-
disjoint. In particular, institutional and disciplinary divides can result ercise in co-creation [6], complicated by dependencies between teams
in the partitioning of early-stage design tasks — such as data profiling, and differences in their competencies. Large diverse teams make it
ideation, creating mockups, and prototyping — and development tasks possible to create, deploy, and provide long-term support for complex
like implementation, testing, deployment, and maintenance. visualizations. However, this division of labor reveals a variety of new
In practice, disciplinary divides between designers and developers design handoff and iteration challenges, which can be compounded by
can be stark [39]. The interaction design literature has examined the the data-driven nature of visualization design.
divide between designers and developers from a number of angles,
including: how designers and developers align their work in collabo- 3 OVERVIEW OF V ISUALIZATION D ESIGN P ROJECTS
rations [11], how they work remotely [65], and how design tools can
function as boundary objects that mitigate designers’ lack of “material” Our reflections on handoff in visualization design and development
experience with software [49]. Recent work by Maudet et al. [39] are anchored in our own experiences as members of a design team
has drawn attention to design breakdowns in design handoff, in which on five large data visualization design projects (Figure 2) conducted
potential disconnects between designers and developers are highlighted between 2012 and 2019. Each project was intended for wide-scale
public release and involved between six months and several years of
1 Data experts had technical and/or domain-related data expertise. data characterization, design, and development work.

2
 To appear in IEEE Transactions on Visualization and Computer Graphics

DATASETS SUMMARIES OF VISUALIZATION DESIGN IMPLEMENTED

Artifacts VISION & GOALS DATA CHARACTERISTICS DOCUMENTATION VISUALIZATIONS

VISUALIZATION DESIGN
Project PROJECT DATA Data Presentation Design VISUALIZATION DEPLOYMENT
CONCEPTUALIZATION CHARACTERIZATION DEVELOPMENT AND USE
Phases Mapping
Interaction Design

Ite Int Data

ng g
ng

ns y
pi tin
nt

tio it
le al di

ra egr
ra ep g
ns e
se g

s
ap a
an o

te D tin

ro in ng
Ca tin

io nd
al ic an
s

M nic
Ch g t
ge
Challenges

Ac app ervi
In ata- ula
Ch chn rst

ct e
ge ipa

s
ta tin

ta u
Da omm

ss g
Te nde

D rtic
es
Da dap

Ed ntic

M res
ng

A
U

P
A

C
C5
C2
C1

C4

C6
C3
Fig. 3: Stages of a data visualization development process and the dependencies between them. We focus on the data characterization, design,
and development phases, highlighting the artifacts that bridge these phases and the challenges (C1-C6) that emerge within them.

For each project, the work was directed by an outside client who tools, spreadsheets, or code, processing data (including text mining),
was also the data provider. Our multi-member design teams, which and understanding specific data types (for example, linguistic analysis
included a rotating cast of designers, visualization researchers, post- of text data). All team members needed to create and understand
docs, graduate students, and interns, were responsible for the majority data mappings from data to visual representation, which included
of the data characterization and visualization design. In all projects, at varying degrees of ideation, basic perceptual understanding, and apply-
least one (and typically more) of the authors participated in the process ing knowledge of visual variables. The project also required people
directly as members of the design team. A separate development team with visual design skills who could design graphics, page layout, and
was tasked with creating, deploying, and providing initial maintenance typography while keeping accessibility in mind. The team included
for the final web-based applications. people with expertise in interaction design, including prototyping and
animation. Likewise, some team members developed visualization
3.1 Projects prototypes to verify and demonstrate designs and engineered the tech-
Energy Visualization for the Inter-American Development Bank nically complex portions of design documents. All team members
(IDB). The earliest of the projects, conducted between 2012–16 with needed to collaborate and communicate with the data provider and
the Inter-American Development Bank produced a suite of visualiza- development team.
tions showcasing energy source generation, import and export, trans- During the Inter-American Development Bank project, the design
mission, and consumption for countries in the Americas, as well as team consisted of one primary visualization design researcher and three
other benchmark countries. The resulting visualizations were hosted Visual Arts students (one undergraduate and two graduate students).
publicly from 2013–18, but are no longer accessible as of 2019. In this The development team consisted of one primary computer science
project, the design and development teams were more closely integrated researcher and one doctoral student. These teams worked closely
than in the other projects, with both playing a substantial role in data together in an iterative fashion and were located on the same campus.
characterization, design, and development. In the remaining projects, the design and development teams were
Energy Futures. This project, conducted over 4 months in 2016, led to separate. The design team consisted of two primary investigators (visu-
the development of four visualizations based on forecasts of Canadian alization researchers), one project coordinator, one design researcher,
energy production and consumption [8]. A second 7-month itera- 1–2 postdoctoral visualization researchers, 2–3 undergraduate or re-
tion [32] of the project in 2017 added a fifth visualization showcasing cently graduated computer science students, 0–1 information design
changes in projected energy demand across the Canadian provinces and undergraduate students, and 1–4 full-time employees with roles in
territories. The visualizations are publicly available at https://apps2.neb- design, development, and specialized data understanding. The devel-
one.gc.ca/dvs. opment team consisted of anywhere between 5 and 9 members of a
Pipeline Incidents. Developed during 2017, this 8-month project professional software development firm located in the same city. In all
produced an interactive visualization system that supported visual ex- projects, the data providers were from separate institutions and were
ploration of incidents that occurred on or around federally-regulated physically separated from the design and development teams.
pipelines. The visualization is publicly available at https://apps2.neb-
one.gc.ca/pipeline-incidents. 3.3 Analysis and Synthesis Process
Energy Imports & Exports. Another similarly-scoped project con- We identified the data-related challenges described in this paper via an
ducted over 16 months in 2017–18 involved creating a set of five ongoing process of reflection [44] through which we worked to refine
visualizations showing historical imports and exports of various energy our design team’s work and communication practices. During each
products from Canada. The visualizations are publicly available at project, we kept records of artifacts produced for meetings, data explo-
https://apps2.neb-one.gc.ca/imports-exports. rations, ideation sketches, planning timelines, and design documents.
Pipeline Conditions. The most recent project, conducted over 18 We also regularly reflected on communication and design challenges
months during 2018–19, focuses on visualizing the conditions placed within our own team. Throughout, we documented and scrutinized
by government regulators on the construction of new pipelines. At the process using approaches drawn from diary-based [15, 48, 53, 56]
the time of publication, this project was near completion, but not yet and autobiographical studies [12, 17, 46]. Individual members of the
publicly accessible. design team, as part of their personal practice, kept notes and images
documenting their work. Later, as part of this autobiographical process,
3.2 Design Team Roles we carefully re-examined our diary-based records and used them to
The members of the design team needed to fulfill a variety of design- identify gaps and challenges.
related roles. The project needed team members who could charac- We also took steps to formalize our design communication processes.
terize data by wrangling data, exploring data in existing visualization Within the design team, we leveraged our initial observations to create

3
shared tools and processes for tracking the team’s progress. During the tion. In this process, we relied heavily on hand-sketching and manual
implementation phase of each project, we also held face-to-face design illustration in tools like Adobe Illustrator [1]. In some cases, we used
review meetings with all stakeholders present. Finally, after each of the chart generation tools and utilities (including RAWGraphs [40] and
three later projects, we organized formal process discussions with the Color Brewer [25]) and hand-coded prototypes using libraries such
data provider and with members of both the design and development as D3 [10]. We developed the presentation design — the overall size
teams to help improve coordination for subsequent projects. We took and layout of the visualization — primarily using Adobe Illustrator [1].
detailed collaborative notes at all meetings. Furthermore, we developed the interaction design using various tools
Based on these reflections, we focused increasing energy across the including paper prototyping [52], textual and sketched descriptions,
remaining projects on improving design communication both within and general-purpose interaction prototyping tools (such as Axure [4]
and across the teams. As part of this effort we documented meetings and Atomic [3]). The final development document was initially in PDF
and design processes using detailed personal records, team records, form but later evolved to be a web-based document. The most recent
design handoff documents, and handoff document revisions. We teased design document was based on Idyll [14], which allowed us to combine
apart the details of the challenges presented in this paper by drawing mockups, coded prototypes, and explanatory text in a single document.
on these detailed records. In discussing a particular challenge we could Visualization Development. This phase was led by the software de-
rigorously re-examine the time-stamped process by which each design velopment team. As the design team, our role was mainly reactionary:
was created, handed off, implemented, re-discussed, re-implemented, we responded to questions about the design, suggested redesigns when
and ultimately released. issues arose, and verified that the implementation was functioning as
Throughout our reflection, we noticed that a number of the recurring intended. Most of the discussions in this phase were grounded in the
handoff challenges were not merely interaction design issues (like those design documentation and in increasingly polished iterations of the
documented by Leiva et al. [35]), but were instead rooted in the deep implemented visualizations.
dependence of the designs on data. From these reflections, we have Deployment and Use. As the visualization was deployed for public
synthesized the most prominent unresolved data-related challenges and use, the software development team was tasked with its maintenance,
illustrated them using real examples from our projects. Where possible, including implementing quarterly data updates. The design team was
the initial reflection was drafted by the team member who most closely involved if a data update contained unexpected values that were not
experienced the example issue. supported by the existing design.

4 V ISUALIZATION D ESIGN & D EVELOPMENT 5 C HALLENGES WHEN D ESIGNING WITH DATA

One outcome of our reflection on the processes and communication in Based on our reflection and observation, we describe six gaps in the
our design projects is a formal model of the major phases of our design data visualization design process.
projects (Figure 3). This model was born out of a need for a vocabulary
to use when coordinating with multiple parties, and serves as a useful C1. Adapting to Data Changes
anchor for the design and communication challenges we discuss in the Data updates can have cascading effects on the data characterization,
remainder of this paper. In a designer-as-developer scenario, a single visualization design, and development phases because all aspects of
person or small team might carry out all of these phases with less need visualization development depend upon the data. The impact of such
for a formal process. In contrast, in our scenarios — which largely effects may not be clear to data providers.
involved multiple teams who did not share a daily working space — In our experience, data is rarely available in its full and final form be-
this model emphasizes the distribution of roles and responsibilities. fore the visualization development process begins, often necessitating
In addition, it highlights the kinds of artifacts that can often facilitate data updates later in the process — sometimes even post-deployment.
communication across phases. Data changes are particularly impactful if they change the data charac-
Project Conceptualization. This phase occurred on the client side, terization or the data used to generate views in the visualization design.
prior to the direct involvement of the design team. The client provided Even when a data change is seemingly innocuous and does not change
the design team with the vision and goals for the project as well as the overall data mapping, it may affect the implementation stage, par-
a dataset. The handoff of these artifacts ranged from simple emailed ticularly where server-side mechanisms for loading, aggregating, or
delivery to more involved full-day workshops between client-side data preparing data have already been established. For example, changing a
experts and the design and development teams. column name or unit symbol may break existing data parsers.
Data Characterization. In this phase, the design team explored and Data updates are not necessarily undesirable. They might provide
characterized the data, prioritizing analyses motivated by the project corrections or additional data, and they might reflect a positive evolu-
vision. This included understanding data types, amounts, and extrema, tion in how the data provider releases data for the visualization. This
as well as the relationship between the data and the project goals. In evolution might itself be prompted by witnessing the interim results of
some cases, the design team recommended a more focused dataset for the visualization design process. As such, the challenge is not to avoid
the visualization, a process which sometimes entailed several iterations data updates altogether but to be able to cope with them efficiently.
with the client-side data team. This phase typically involved some For example, late in the process of designing visualizations for the
data wrangling [29], but was more akin to exploratory data analysis Inter-American Development Bank project, the design and development
or domain problem characterization [45]. We used a number of tools teams had created a mature, late-stage visualization design of energy
to support this phase, including spreadsheets, hand-coded scripts, and generation data from Latin American countries (Figure 4-top). Up
visualization exploration software such as Tableau [58], as well as hand- to that point, all design decisions had been made on the basis of the
sketching for preliminary ideation. This culminated in the creation of team’s work with the initial data that was provided by the client. This
a summary of data characteristics, which we typically delivered to characterization led to a design that arranged data about different energy
the client-side data experts via an in-person presentation. The insights sources on a circle, showing the relationship of energy inputs (top half
and shared understanding gained throughout this phase served as a of the circle) to energy generation and losses (bottom half).
foundation for future design work. At this stage, an update added data for benchmark countries like
Visualization Design. The design phase encompassed the abstraction China, creating views (Figure 4-bottom) in which a single energy source
and encoding/interaction design phases of the nested model [45] to- (in this case coal) visually overwhelmed the values from other sources.
gether with partial algorithm design and extensive visual presentation This dramatically altered the form and legibility of the visualization,
design. This was a two-stage process. First, we developed a visualiza- crowding all of the original data onto a small slice at the side of the
tion concept and received initial approval from the client. Then, we chart, and overlapping the arcs and labels. Given the late stage of the
refined and polished the final design and documented it via a visual- design process, there were not enough resources available to iterate
ization design documentation shared with both clients and developers. the design, and the resulting visualization was quite different from the
We developed a data mapping on the basis of the data characteriza- original concept.

4
 To appear in IEEE Transactions on Visualization and Computer Graphics

INITIAL DESIGN FILTERS APPLIED UPDATED DESIGN

MATURE VISUALIZATION DESIGN
(Data for all Latin American Countries)
(EDGE CASE)

Some combinations of
filters resulted in
edge cases in which
bars broke the layout.

A data update late in

development added data
for countries like China
where the heavy use
of coal left all other
In response,
categories compressed.
designers added
(+Data for Baseline Countries)

non-linear scale
AFTER DATA UPDATE

breaks to bars.

Fig. 6: The initial design of the Energy Imports & Exports visualiza-
tion (left) responded poorly to particular combinations of filters (center)
and ultimately required a revision (right).

Fig. 4: Adding data from additional countries late in the design of this can be tested relatively early in the development process and designs
visualization resulted in a single energy source (coal) dominating the can be adjusted as needed. However, when design and development are
view which made other energy sources difficult to compare. separated, designers do not always have the tools or skillsets to fully
test all possible combinations of inputs. As such, potential problems
might only be uncovered during the development phase after many
Late-stage data updates can also cause subtle changes to how a design decisions have been finalized. At this stage, any changes to the
visualization is perceived. For example, in the Energy Futures project, a design can incur significant design and development work, limiting
visualization of demand shares by energy source was initially designed the possible ways in which the visualization design can be adapted to
using data based on the energy production stages. The design team mitigate the problems.
characterized the data and moved on to the design phase, creating a In the design of the Energy Imports & Exports visualizations, the
D3 prototype (Figure 5). Our focus here was to support comparison design team developed a mirrored chart that could be filtered to show av-
between provinces that have order-of-magnitude differences in scale. erage quarterly electricity prices between any combination of US states
Later, the dataset was changed to one based on end-use energy demand. and Canadian Provinces (Figure 6-left). This data mapping worked
The characteristics of both datasets were quite similar, so the design well for the vast majority of views, including the various combinations
work continued with attention shifting to other concerns. However, of test data that the design team used when creating their initial docu-
there was a key but subtle difference in the new dataset: nearly all ments. However, once this design was implemented it became clear that
renewable energy was included within the Electricity category rather filtering by particular combinations of states and provinces revealed
than in the Renewables category. The Renewables category in the outliers which had been masked in the initial samples (Figure 6-center).
original dataset was already quite small, so this change went unnoticed. The design solution was constrained by the fixed size and very lim-
While the resulting visualization still shows the data accurately, it can ited space available for the bar chart, as well as by the need to maintain
be easily misinterpreted because it does not clearly communicate the consistency with the data mapping in other parts of the visualization.
fact that the Electricity category also includes most renewables. Reconfiguring other parts of the visualization was also not feasible late
in the development stage. Ultimately, the design team opted to use a
C2. Anticipating Edge Cases compressed scale break for these outliers (Figure 6-right). This solution
It is difficult for designers to anticipate and test all possible combina- makes it impossible to make direct visual comparisons between large
tions of interactive inputs that a visualization might receive. As a result, values above the scale break and reduces the visual impact of large
it can be hard to anticipate situations in which a chart design or data values, but still communicates relative differences in scale within the
mapping may break. available space and minimizes changes in other parts of the visualiza-
Common visualization interactions such as filtering or aggrega- tion. Several related scale issues also emerged late in the development
tion essentially change data mappings in real time. In design-as- of the Pipeline Conditions visualization, necessitating new design work
development scenarios with live prototypes, these kinds of interactions during implementation. In all cases, if the edge cases had been iden-
tified earlier in the design process, the entire design might have been
conceived differently.
EARLY DESIGN EDITED EXPORT IMPLEMENTED VERSION C3. Understanding Technical Constraints
A late-stage Designers may not be aware of all of the technical constraints and
change to the challenges that can occur during the development phase. This can
PE dataset altered lead to uncertainty about design feasibility and can also trigger more
the meaning of
categories (like dramatic revisions during development.
renewables). It is not always clear what types of software and hardware limitations
may pose challenges for a design. Visualization designers tend to
focus considerable attention on the choice of visual mappings and on
Fig. 5: Changes to the dataset for this Energy Futures visualization providing a useful and appealing interaction experience. Web and
altered category meanings, creating potentially misleading values. application developers, meanwhile are more likely to be tasked with

5
 PRE-INTERACTION MOCKUP
Accessibility libraries
added rectangular selection Creating a second mockup
indicators around even that introduces a new column
non-rectangular controls.
requires manually adding a
large number of complex
data-driven elements.

POST-INTERACTION MOCKUP
Fig. 7: A portion of a visualization showing a heat map of Canadian
energy exports to United States Petroleum Administration Defense Dis-
tricts (PADDs). Due to confusion surrounding the limitations of an ac-
cessibility template, color mappings and the layouts of non-rectangular
components in this visualization needed to be adapted to accommodate
rectangular selection indicators.

delivering robust, efficient, standards-compliant implementations of a

design. While collaborators often have an appreciation for the others’
areas of focus, it can be difficult to be aware of all potential issues
that the other group faces. From a design perspective, this can lead
to uncertainty, particularly when proposing unique or unconventional
designs. Across our projects, the designers often tried to mitigate
this uncertainty by prototyping and testing novel pieces of the designs Fig. 8: Two views of a parallel sets-style visualization of incidents on
in code. However, we still frequently encountered technical hiccups pipelines and pipeline facilities. Top: A single category is selected.
during development. Highlighted gray lines represent how the incidents in the selected
For example, during the design of the Pipeline Conditions visual- category relate to incidents in the categories of the column to the right.
ization, the design team created detailed working versions of several Bottom: The design allows additional columns to be dragged into
complex components, including an interactive keyword browser that the visualization from the right. This single interaction introduces
leveraged a third-party physics engine. However, the development team substantial complexity to the resulting view. Prototyping, testing, and
worked within a different set of constraints that included cross-browser communicating this resulting view in a data-accurate way is difficult
compatibility, future code maintainability, and a decision to use a dif- using conventional graphic and interaction design tools.
ferent underlying web framework to implement the site. As such, they
chose to re-implement these components from scratch. This resulted
in controls that were superficially similar to the original designs but
that behaved quite differently. As a result, they required substantial data, a different transformation of the data, or a different encoding of
additional refinement to achieve behaviour that was already present in the data. For instance, filtering operations reduce the set of data being
the design team’s prototypes. considered. This can result in a change of position or appearance of the
Often, these issues arise not from limitations in the visualization remaining elements. Similarly, each brushing and linking interaction
libraries themselves, but in visualization-adjacent aspects of the im- requires a change to the encoding applied to a very specific set of marks
plementation such as data loading, cross-browser compatibility, and spread across several views. When prototyping these interactions by
accessibility requirements. During the development phase of the En- hand or using graphic design tools, designers must manually manage
ergy Imports & Exports project, the design team was surprised to learn and update large numbers of individual data elements. Modeling the
that the government-mandated accessibility template within which the transitions between such views is even more difficult, especially when
design would sit also applied to individual components of the visualiza- complex animations are required.
tion. This meant that each selectable element of the visualization would Transitions like the one in Figure 8 are particularly challenging.
be surrounded by a stroked rectangular box with a dominant color. In this example, drawn from our Pipeline Incidents project, a simple
This late discovery resulted in a mismatch between the visual aesthetic interaction with the flow visualization can add an additional column,
of the design, which was built around hexagonal tiles, and the bright introducing dozens or even hundreds of new arcs. The complexity
halos produced by the accessibility template (Figure 7). The use of that this single interaction adds to the view is substantial. Multiple
the template also forced the design team to revisit the data mapping to forking curves appear with varying positions and thicknesses, all of
ensure that the color used by the accessibility overlays did not conflict them related to the previous selection.
with the palette used to encode import and export data. Creating a data-accurate version of the resulting view is very dif-
ficult using conventional graphic design tools and requires manually
C4. Articulating Data-Dependent Interactions computing the size, positions, and connectivity of large numbers of
When ideating and specifying new data-dependent interactions, design- edges, then manually adding them to the mockup. This expense is
ers often need to generate a variety of different data-accurate views multiplied every time there is a change to the underlying design or
showing the visualization in multiple states. This extra cost can make an update to the data. Even trivial changes such as changing screen
data-driven interactions challenging to develop and communicate to dimensions or color schemes can require manual revisions to these
other team members. designs. Specifying transitions and animations between these views is
The outcome of most interactions with a visualization depends on the also difficult, even when using interactive wireframing and animation
data itself. Typically, these data-dependent interactions generate new tools, since they do not include data binding or animation support for
views of the visualization that may represent a different subset of the such fine-grained elements. Finally, because this view represents only

6
 To appear in IEEE Transactions on Visualization and Computer Graphics

size encoding expected, but did not precisely specify, that “size” would be interpreted
rendered as
diameter (incorrect) as area rather than diameter. This expectation drew from their own
domain knowledge of common visualization guidelines. However, this
guideline is not inherently obvious, particularly to non-experts. In this
case, the initial implementation mapped the data to the circles’ diameter
rather than their area. This subtle difference in the mapping was diffi-
Misinterpretations of cult to detect via visual inspection alone (see Challenge C6–Integrity).
data mappings in design
documentation can be
Ultimately, the error was caught by happenstance. However, once the
subtle and hard to detect. discrepancy was noticed, extra development time was needed to change
the mapping to what the designers originally intended.
Another example from the Pipeline Conditions project illustrates
that data mapping specification is challenging even in face-to-face
size encoding conversation. One portion of this visualization displayed a horizontal
rendered as scrolling list of projects (Figure 10). On either side of this list a bar
area (correct)
denoted the number of projects to the left and right. At one face-to-
face design review session, the issue was raised of how best to map
Fig. 9: A static mockup of a bubble chart visualization. While tooltips the number of remaining projects to the size of the bars, as the space
on the bubbles provide precise values, the image alone does not contain allotted for the bars was quite small. A simple solution was agreed
enough information to enable a developer to re-create it. In particular, upon verbally. However, the next implementation cycle revealed that,
it does not specify that the data values shown are mapped to the area, in fact, even this simple solution could be interpreted in multiple ways:
not the diameter, of the circles. one team understood that the maximum size of the bar would be set
based on the entry with the largest number of possible items, while
the other team understood that the maximum size of the bar would
one of many possible application states, exploring the effectiveness of represent a fixed value. This confusion ultimately required a third set
the interaction requires replicating the process for other views. of follow-up discussions in order for the teams to reach a consensus.
Unfortunately, exploring this same interaction by coding a low-
fidelity interactive prototype capable of supporting it also entails con- C6. Preserving Data Mapping Integrity across Iterations
siderable effort. In comparison to graphic design tools, low-fidelity Because it is difficult to systematically compare implementations
interactive prototypes also make it considerably more difficult to ex- against design documents, there is a serious risk that misinterpretations
amine alternative layouts, typefaces, controls, and other aspects of the or misapplications of the data mapping can go unnoticed during and
design. Testing pixel-perfect versions of interactions and transitions in after development.
a coded prototype effectively requires committing to and implementing
As a design is implemented, differences can emerge due to a variety
the entire design.
of factors including bugs, data updates, inconsistencies in the initial
Across the five projects, we often took both approaches, using
designs, and misinterpretations of the data mapping. This is in part
graphic design tools to sketch and visually polish components and
because the implementation is separate from the design documentation.
visualization views, while simultaneously implementing prototypes to
The primary method of testing the implementation’s adherence to the
test the impact of the interaction. We also used interactive charting
design is visual inspection and comparison to the original specification.
tools like Tableau [58] and RAWGraphs [40] to create data-accurate vi-
Furthermore, every new iteration of an implementation typically intro-
sualization elements that could then be exported back into graphic and
duces numerous small changes. This can make it difficult to manually
interaction design tools to create richer mockups. However, the gaps
keep track of what has and has not been inspected or to know when
between each of these tools is considerable and interaction prototyping
it is the right time to flag an issue. Even if all parties recognize the
consumed a substantial amount of the design team’s bandwidth.
importance of preserving the data mapping, visual inspection of the
implementation alone may not reveal misinterpretations of the design
C5. Communicating Data Mappings
or of the data mapping at any given iteration.
Implementing a data mapping correctly requires more detail and preci- The example in Figure 11 shows a comparison between a small
sion than can be easily inferred from a mockup of a visualization view. portion of a visualization design document and an implementation of
However, precise and complete specification of data mappings is not that design. Many small differences related to both presentation design
well-supported by current design tools. and the data mapping are apparent, including issues with the typography
The mapping from data to visual representation is the most funda- and alignment. In addition, the set of blue arrows in the legend point
mental aspect of a visualization. As such, it is critical that a designer up, but should point down to align with the downward export arrows in
creating a visual mapping be able to communicate their intent to others the rest of the visualization.
on the team, especially developers. Current options for communicating The incorrectly-sized bubble charts described in Challenge C5 (Data
data mappings are limited and must be created manually. A static Mappings) also illustrate the difficulty of noticing mistakes in imple-
rendering of the view may seem sufficient if the visualization is de- mented designs. While detecting this kind of error using only visual
signed to be easily read by non-experts. However, example views inspection is already difficult, it becomes even more challenging when
may not capture many of the nuances and details that are important
for implementation purposes. For example, correctly mapping a data
value to a visual mark often requires data transformations or lookups,
which may involve multiple hidden steps (such as using a classifica-
Dark bars in this horizontal list show
tion algorithm to bin heat map values). Furthermore, the complexity the number of projects to the left
of the data may make it difficult to explicitly show all data cases in and right of the currently visible set.
the provided views, particularly where interaction is concerned (see
Challenge C4–Interactions).
One clear example of this issue emerged late in the Energy Fu-
tures project, which included the bubble chart visualization shown in
Figure 9. The visualization design documentation outlining this visual-
ization included one static view of the bubble chart created in Adobe Fig. 10: The mapping between number of projects and the width of the
Illustrator [1], together with information about which data columns bars in this horizontal list was unclear both in the design document and
were to be mapped to the size of the circles. The visualization designers in follow-up conversations.

7
 DESIGN DOCUMENT IMPLEMENTED SYSTEM data discovery tools like DataTours [42] may provide a useful starting
a
point. Similarly, visual tools for quickly comparing the distribution of
values in different datasets might help designers more readily detect
Font size & text differ problematic changes without relying on statistical summaries [38].

6.2 Design Phase

Data-Driven Visualization Ideation. A number of the challenges we
experienced are associated fundamentally with the challenge of ideating
data-driven visualization designs. Existing commercial tools for man-
Alignment not maintained ual vector-based graphic design such as Adobe Illustrator [1] have little
support for creating complex, data-driven views, while visualization ex-
ploration and generation tools such as Tableau [58] or RAWGraphs [40]
have limited support for custom visuals and interaction. Meanwhile,
programming tools and lower-level libraries can be challenging to use
as rapid ideation platforms and can disempower non-programmers. For-
tunately, recent projects like Data Illustrator [37], Data Ink [64] and
Data-Driven Guides [30] highlight the potential for more expressive
Arrow direction inverted data-driven graphic design tools. Several of the challenges we experi-
enced emphasize the need for further work in this space. More direct,
dynamic, and expressive tools for designing with data could facilitate
Fig. 11: Detailed differences between a piece of a visualization de- rapid exploration of different design alternatives even in the face of
sign as specified in the original design documentation (left) and an changing data (C1–Changes). Similarly, more rapid exploration of data-
implemented version of that design (right). Manually inspecting each driven design alternatives could make it easier to discover unexpected
iteration of the developed visualization to compare against the design edge cases (C2–Edge Cases) and prototype data-dependent interactions
document is tedious and error-prone. (C4–Interactions). Mei et al. [43] identify several additional research
directions for these kinds of tools, including supporting refinement
based on existing visualizations, providing better debugging support,
there are frequent data updates during the visualization development and exploring programming for dynamic data and interaction.
process. Frequent changes and updates like these can lead to uncer- Where artifact creation cannot be unified into one single authoring
tainty about whether the visualization matches expectations due to an platform, better handoff tools may also offer opportunities for designers
incorrect mapping or simply due to differences in the data. and developers to synchronize the artifacts across multiple systems.
Already, tools like Hanpuku [7] have explored bridging the graphic
6 D ISCUSSION design expressivity of Adobe Illustrator and the data-driven prototyping
capabilities of D3 [10]. However, designing bi-directional workflows
The challenges we have highlighted stem directly from the intrinsic between these sorts of existing tools usually entails compromise —
connection between visualizations and the source data that drives them. often intersecting the limitations of each tool and limiting the pieces of
When compared to other kinds of interfaces, data affects the visibility functionality that can be translated. This suggests that unidirectional
of elements, their layout, and their appearance to a much greater extent. handoff tools, like those now widely used in interaction design, are a
Viewers of these tools interact not only with the interface but with the more likely first step.
data. As a result, developers and software engineers must also deal with Data-Driven Interaction Prototyping. Prototyping data-driven inter-
the pragmatic limitations of the datasets when considering performance actions within a visualization design is important for exploring different
and interactive capabilities. Yet, for designers, anticipating all of the interaction options, ensuring the scalability and understandability of
implications of scale and interaction for a given dataset remains chal- those interaction options, and communicating interaction designs to
lenging. This is magnified by the reality that datasets are likely to be developers (C4–Interactions). Yet data-driven interactions can be com-
updated many times both during and after the design process. Together, plex to prototype because one interaction can simultaneously cause
these challenges suggest a variety of opportunities for research and tool a change to a large number of data-driven elements in a design. Un-
creation that could specifically support the visualization design process, fortunately, commercially-available user interface prototyping tools
both for individuals and for collaborative teams. do not address this challenge. Commercial interaction design tools
make it straightforward to prototype interactions and transitions using
6.1 Data Characterization static mockups, and even provide some limited support for data-driven
The process of exploring and characterizing new datasets in preparation prototyping — for example by populating user profiles or lists of infor-
for visualization design work has much in common with data analysis, mation. However, they provide little support for creating visualization
and therefore many existing data analysis tools can be appropriated views whose layout, appearance, and interactivity are all deeply and
within this space. However, Challenge C1 (Changes) illustrates some inherently driven by data. As such, opportunities remain for new tools
opportunities to create data characterization tools and processes that that allow designers to more expressively prototype and test potential
are dedicated specifically to visualization design. Changes to data interactions, either by bootstrapping on top of existing visualization
characteristics can have a substantial impact upon the final visualization tools or via new authoring interfaces.
outcome because visualization design choices typically reflect the shape Data Mapping Documentation. Communicating and documenting
and parameters of the given dataset. However, the extent to which new design intent is useful not only for explaining visualization designs to
data may impact the robustness of the visualization may not be clear a development team, but also when communicating with other team
during the data update process. members or stakeholders and when producing project documentation.
Data characterization tools could help remove this disconnect by One opportunity highlighted by challenge C5 (Data Mappings), is
helping designers better understand how the data has changed from one to design tools and methods for communicating data mappings. Such
version to the next and how those changes might alter the design of tools would support explicit communication of the relationship between
the visualization. This could include tools for highlighting changes in data structures and their graphical representation with enough detail to
data column names, extrema, and statistical distributions of data, or convey any transformations, calculations, and algorithms required.
for simulating likely future changes based on the current distribution Related work on visualization grammars [54, 57, 62] provides a
of values. Recent work on semi-automated approaches for outlier useful starting point, as do projects that represent the visualization
detection and profiling in data mining toolboxes like Orange [16] or pipeline [59] and support the deconstruction and modification of data

8
 To appear in IEEE Transactions on Visualization and Computer Graphics

mappings for existing visualizations [24]. However, there remains a from phase to phase in very discrete stages and the need for clear com-
need for data mapping tools that are accessible to designers without munication and knowledge transfer between teams was very strong.
programming skills but which still communicate the nuances of a data However, the challenges we identify are more closely related to the
mapping in enough detail to reproduce it programmatically. way that data permeates every aspect of the design process and would
Data Visualization Design Documentation Although data mappings remain, to some extent, regardless of team configuration. As we have
are fundamental to a visualization, they are only one part of its design. shown, data impacts nearly every facet of visualization design. A single
Ultimately, any data mapping must be communicated as part of a larger update to the data distribution can completely change the effectiveness
body of design documentation that also captures graphical presentation of a visualization (C1–Changes). Incorporating standard data interac-
(including layout, typography, and color) as well as interaction. While tions into a design can increase the number of application states that
currently this documentation is often ad-hoc and informal, more sys- need to be considered and make it much more difficult to find edge
tematic tools for capturing and communicating design details could be cases (C2–Edge Cases). Moreover, the fundamental dependence of the
valuable in larger visualization design projects. Given the highly visual visualization on data may require a robust and complex backend imple-
and interactive nature of visualization designs, one basis for these kinds mentation, the constraints of which are not always apparent during the
of documents could be explorable explanations or other interactive doc- design phase, making it difficult to anticipate technical challenges or
uments. This format, popularized by Bret Victor [61] is increasingly provide coded prototypes that can easily be translated to final imple-
used in data analysis practice, and recent literate programming tools mentations (C3–Constraints). Prototyping interactions with data can
like Observable [47], litvis [63], and Idyll [14] may provide promising also involve tedious calculations and significant changes from view to
platforms for creating and documenting visualization designs. view (C4–Interactions). Likewise, articulating and testing the mapping
Annotation tools can also play an instrumental role in design com- from data to its visual representation is a fundamental and complex task
munication, particularly when aspects of a design may not be obvious that is not well-supported by conventional user interface design tools
from the visual artifacts alone. Already, annotation is often present (C5–Data Mappings, C6–Integrity). Others before us have articulated
in authoring tools for low-fidelity user experience prototypes (cLus- difficulties in design handoff, including articulating edge cases [35].
ter [50], D.Note [26], SketchComm [36], SILK [33]). Going forward, However, everything in the visualization design — the mapping, the
visualization-specific annotation libraries like ChartAccent [51] have graphic design, and the interaction design — depends deeply upon the
the potential to enable richer data-driven selection and annotation that data. Therefore, the data adds complexity to every step of this process.s
could also support visualization design documents. Finding and addressing opportunities to support that complexity could
make the visualization design process more effective, more expressive,
6.3 Development Phase and more accessible to people with a variety of skillsets.
As emphasized by challenge C6 (Integrity), it is important that anyone
testing an implemented visualization against its design documentation 7 C ONCLUSION
is able to identify discrepancies between the intended design and the Based on reflections on our experiences as designers during the data
implementation. Moreover, they should be able to differentiate between characterization, visualization design, and development phases of sev-
discrepancies that are due to data differences, discrepancies that are due eral large-scale collaborative visualization projects, we have highlighted
to incomplete implementation, and unintentional discrepancies that are the increased complexity that data fundamentally introduces to the de-
due to miscommunication or mistakes. The most vital discrepancies sign process. Data-related challenges span all phases of design and
are those pertaining to data mappings (C5–Data Mappings). However, include adapting to late-stage data changes, anticipating edge cases, ar-
presentation discrepancies related to screen size, alignment, layout, ticulating data-dependent interactions, communicating data mappings,
typography, and non-data color issues like those illustrated in Figure 11 and preserving data mapping integrity in implementation. These point
also play a role. In these cases, new tools for supporting differencing to several opportunities to create tools that directly support the visual-
and visual comparison between visualizations [20, 21] may be partic- ization design process through specific data-related features. Creating
ularly valuable. Interaction discrepancies, on the other hand, may be these more powerful tools could make the design process more robust,
more difficult to detect, but are especially important when they can efficient, and accessible to people in a variety of design roles.
affect the understanding of the data. Working visualization prototypes
that simulate the final interactions can mitigate some of these issues, ACKNOWLEDGMENTS
but only when the technical constraints of the design and development We thank (alphabetically): Amanda Harwood, Annette Hester, Ryan
teams are aligned ahead of time (C3–Constraints). Hum, Katherine Murphy, and Peter Watson and their teams at the Na-
tional Energy Board of Canada (NEB) for facilitating and supporting
6.4 Many Paths to Visualization Design this project. We also thank the software development team at Viz-
The challenges for data visualization design and handoff described worX. Finally, we thank members of the design teams who took part
here are a direct reflection of our own experience in a large, multi-team in the projects that led to this paper (alphabetically): Bon Adriel Ase-
environment and are limited to our perspective as the design team. With niero, Peter Buk, Shreya Chopra, Claudio Esperança, Tina Huynh, Lisa
the exception of the first project, the physical and temporal separation Hynes, Lindsay MacDonald Vermeulen, Claudia Maurer, Charles Perin,
between data compilation, visualization design, and development were Richard Pusch, Lien Quach, Katrina Tabuli, Markus Tessmann, Sarah
relatively high. Not all visualization projects are configured in this way. Storteboom, Sonja Thoma, and Jo Vermeulen.
Some projects involve multiple co-located people in highly specialized This project was funded in part by The National Energy Board of
roles (for example in newsrooms [19], where journalists, data analysts, Canada, the National Sciences and Engineering Research Council of
designers and programmers might work closely together on very tight Canada, Alberta Innovates Technology Futures, SMART Technolo-
timelines). Meanwhile, many projects still involve individual designer- gies, and the European Unions Horizon 2020 research and innovation
developers taking on many roles simultaneously. There is a wealth of programme under the Marie Sklodowska-Curie grant agreement No.
literature discussing managing communication on design teams [18], 753816.
including work on developing shared mental models of tasks, teams,
and processes [5,28], sharing meaning-making activities [34], and man- R EFERENCES
aging the organization of design work [13]. As such, it is clear that [1] Adobe, Inc. Adobe Illustrator. http://adobe.com/products/illustrator/.
technology is only one way to help mitigate communication challenges [2] Adobe, Inc. Adobe XD. http://adobe.com/products/xd.html.
and that processes, communication frameworks, education, environ- [3] Atomic.io Limited. Atomic Prototyping. http://www.atomic.io.
ment, mutual trust/comfort, and increased face-to-face time are all [4] Axure Software Solutions, Inc. Axure RP. https://www.axure.com.
factors that affect the design process. [5] P. Badke-Schaub, A. Neumann, K. Lauche, and S. Mohammed. Mental
The separation of our teams likely amplified and made it easier to models in design teams: A valid approach to performance in design collab-
identify some of the challenges we articulate. Our projects transitioned oration? CoDesign, 3(1), Mar. 2007. doi: 10.1080/15710880601170768

9
 [6] A. Bigelow, S. Drucker, D. Fisher, and M. Meyer. Reflections on How 1145/1753326.1753400
Designers Design with Data. In Proceedings of the 2014 International [27] InvisionApp, Inc. InVision. http://www.invisionapp.com.
Working Conference on Advanced Visual Interfaces, AVI ’14. ACM, 2014. [28] T. E. Johnson, Y. Lee, M. Lee, D. L. O’Connor, M. K. Khalil, and
doi: 10.1145/2598153.2598175 X. Huang. Measuring sharedness of team-related knowledge: Design and
[7] A. Bigelow, S. Drucker, D. Fisher, and M. Meyer. Iterating between Tools validation of a shared mental model instrument. Human Resource Develop-
to Create and Edit Visualizations. IEEE Transactions on Visualization and ment International, 10(4), Dec. 2007. doi: 10.1080/13678860701723802
Computer Graphics, 23(1), Jan 2017. doi: 10.1109/TVCG.2016.2598609 [29] S. Kandel, A. Paepcke, J. M. Hellerstein, and J. Heer. Enterprise data
[8] T. Blascheck, L. M. Vermeulen, J. Vermeulen, C. Perin, W. Willett, T. Ertl, analysis and visualization: An interview study. IEEE Transactions on
and S. Carpendale. Exploration Strategies for Discovery of Interactivity Visualization and Computer Graphics, 18(12):2917–2926, Dec 2012. doi:
in Visualizations. IEEE Transactions on Visualization and Computer 10.1109/TVCG.2012.219
Graphics, pp. 1–1, 2018. doi: 10.1109/tvcg.2018.2802520 [30] N. W. Kim, E. Schweickart, Z. Liu, M. Dontcheva, W. Li, J. Popovic,
[9] Bohemian B.V. Sketch. http://www.sketchapp.com. and H. Pfister. Data-Driven Guides: Supporting Expressive Design for
[10] M. Bostock, V. Ogievetsky, and J. Heer. D3 Data-Driven Documents. IEEE Information Graphics. IEEE Transactions on Visualization and Computer
Transactions on Visualization and Computer Graphics, 17(12):2301–2309, Graphics, 23(1):491–500, Jan 2017. doi: 10.1109/tvcg.2016.2598620
Dec 2011. doi: 10.1109/tvcg.2011.185 [31] R. M. Kirby and M. Meyer. Visualization collaborations: What works and
[11] J. M. Brown, G. Lindgaard, and R. Biddle. Joint Implicit Alignment why. IEEE Computer Graphics and Applications, 33(6):82–88, 2013. doi:
Work of Interaction Designers and Software Developers. In Proceedings 10.1109/MCG.2013.101
of the 7th Nordic Conference on Human-Computer Interaction: Making [32] S. Knudsen, L. Quach, P. Buk, K. Tabuli, S. Chopra, W. Willett, S. Carpen-
Sense Through Design, NordiCHI ’12. ACM, 2012. doi: 10.1145/2399016 dale, J. Vermeulen, D. Kosminsky, J. Walny, M. West, C. Frisson, B. A.
.2399121 Aseniero, L. M. Vermeulen, and C. Perin. Democratizing Open Energy
[12] R. V. Bullough Jr and S. Pinnegar. Guidelines for quality in autobiograph- Data for Public Discourse using Visualization. In Extended Abstracts of
ical forms of self-study research. Educational researcher, 30(3):13–21, the SIGCHI Conference on Human Factors in Computing Systems. ACM
2001. doi: 10.3102/0013189X030003013 Press, 2018. doi: 10.1145/3170427.3186539
[13] M.-L. Chiu. An organizational view of design communication in design [33] J. A. Landay. SILK: Sketching Interfaces Like Krazy. In Conference
collaboration. Design Studies, 23(2), Mar. 2002. doi: 10.1016/S0142 Companion on Human Factors in Computing Systems, CHI ’96. ACM,
-694X(01)00019-9 1996. doi: 10.1145/257089.257396
[14] M. Conlen and J. Heer. Idyll: A markup language for authoring and [34] A. Larsson. Making sense of collaboration: The challenge of thinking
publishing interactive articles on the web. In Proceedings of the ACM together in global design teams. In Proceedings of the 2003 International
Symposium on User Interface Software and Technology, UIST ’18. ACM, ACM SIGGROUP Conference on Supporting Group Work, GROUP ’03.
2018. ACM, 2003. doi: 10.1145/958160.958184
[15] M. Czerwinski, E. Horvitz, and S. Wilhite. A diary study of task switching [35] G. Leiva, N. Maudet, W. Mackay, and M. Beaudouin-Lafon. Enact: Reduc-
and interruptions. In Proceedings of the SIGCHI Conference on Human ing designer–developer breakdowns when prototyping custom interactions.
Factors in Computing Systems, CHI ’04, pp. 175–182. ACM, 2004. doi: ACM Transactions on Computer-Human Interaction (TOCHI), 26(3):19,
10.1145/985692.985715 2019. doi: 10.1145/3310276
[16] J. Demšar, T. Curk, A. Erjavec, Črt Gorup, T. Hočevar, M. Milutinovič, [36] G. Li, X. Cao, S. Paolantonio, and F. Tian. SketchComm: A Tool to
M. Možina, M. Polajnar, M. Toplak, A. Starič, M. Štajdohar, L. Umek, Support Rich and Flexible Asynchronous Communication of Early Design
L. Žagar, J. Žbontar, M. Žitnik, and B. Zupan. Orange: Data mining Ideas. In Proceedings of the ACM 2012 Conference on Computer Sup-
toolbox in python. Journal of Machine Learning Research, 14:2349–2353, ported Cooperative Work, CSCW ’12. ACM, 2012. doi: 10.1145/2145204
2013. .2145261
[17] A. Desjardins and A. Ball. Revealing tensions in autobiographical design [37] Z. Liu, J. Thompson, A. Wilson, M. Dontcheva, J. Delorey, S. Grigg,
in HCI. In Proceedings of the SIGCHI Conference on Designing Interac- B. Kerr, and J. Stasko. Data Illustrator: Augmenting Vector Design
tive Systems, DIS ’18, pp. 753–764. ACM, 2018. doi: 10.1145/3196709. Tools withLazy Data Binding for Expressive Visualization Authoring. In
3196781 Proceedings of the SIGCHI Conference on Human Factors in Computing
[18] C. Eckert, A. Maier, and C. McMahon. Communication in design. In Systems, CHI ’18, 2018.
J. Clarkson and C. Eckert, eds., Design process improvement: A review of [38] J. Matejka and G. Fitzmaurice. Same Stats Different Graphs. In Pro-
current practice. Springer London, 2005. doi: 10.1007/978-1-84628-061 ceedings of the 2017 CHI Conference on Human Factors in Computing
-0 10 Systems - CHI’17. ACM Press, 2017. doi: 10.1145/3025453.3025912
[19] R. Fischer-Baum and C. Esteban. Working in a graphics visual storytelling [39] N. Maudet, G. Leiva, M. Beaudouin-Lafon, and W. Mackay. Design
team. Keynote presentation, Information Plus Conference, Potsdam, Ger- Breakdowns: Designer-Developer Gaps in Representing and Interpreting
many, Oct 2018. Interactive Systems. In Proceedings of the 2017 ACM Conference on
[20] M. Gleicher. Considerations for Visualizing Comparison. IEEE Transac- Computer Supported Cooperative Work and Social Computing, CSCW
tions on Visualization and Computer Graphics, 24(1), Jan 2018. doi: 10. ’17. ACM, 2017. doi: 10.1145/2998181.2998190
1109/TVCG.2017.2744199 [40] M. Mauri, T. Elli, G. Caviglia, G. Uboldi, and M. Azzi. RAWGraphs:
[21] M. Gleicher, D. Albers, R. Walker, I. Jusufi, C. D. Hansen, and J. C. A Visualisation Platform to Create Open Outputs. In Proceedings of the
Roberts. Visual comparison for information visualization. Information 12th Biannual Conference on Italian SIGCHI Chapter, CHItaly ’17. ACM,
Visualization, 10(4), Oct 2011. doi: 10.1177/1473871611416549 2017. doi: 10.1145/3125571.3125585
[22] J. Gray, L. Chambers, and L. Bounegru. The data journalism handbook: [41] S. McKenna, D. Mazur, J. Agutter, and M. Meyer. Design Activity
How journalists can use data to improve the news. O’Reilly Media, Inc., Framework for Visualization Design. IEEE Transactions on Visualization
2012. and Computer Graphics, 20(12), Dec 2014. doi: 10.1109/TVCG.2014.
[23] H. Hagen, D. Keim, T. Munzner, S. North, and H. Pfister. Vis restruc- 2346331
turing recommendations, Fall 2018. http://ieeevis.org/governance/1810- [42] H. Mehta, A. Chalbi, F. Chevalier, and C. Collins. DataTours: A Data
Restructuring-Recommendations.pdf, 2018. Narratives Framework. In IEEE InfoVis 2017 Posters, Oct 2017.
[24] J. Harper and M. Agrawala. Deconstructing and Restyling D3 Visual- [43] H. Mei, Y. Ma, Y. Wei, and W. Chen. The design space of construction
izations. In Proceedings of the 27th Annual ACM Symposium on User tools for information visualization: A survey. Journal of Visual Languages
Interface Software and Technology, UIST ’14. ACM, 2014. doi: 10.1145/ & Computing, Oct 2017. doi: 10.1016/j.jvlc.2017.10.001
2642918.2647411 [44] M. Meyer and J. Dykes. Reflection on reflection in applied visualization
[25] M. Harrower and C. A. Brewer. Colorbrewer.org: An online tool for research. IEEE Computer Graphics and Applications, 38(6), Nov. 2018.
selecting colour schemes for maps. The Cartographic Journal, 40(1):27– doi: 10.1109/MCG.2018.2874523
37, 2003. doi: 10.1179/000870403235002042 [45] T. Munzner. A nested model for visualization design and validation. IEEE
[26] B. Hartmann, S. Follmer, A. Ricciardi, T. Cardenas, and S. R. Klemmer. Transactions on Visualization and Computer Graphics, 15(6), Nov. 2009.
D.Note: Revising User Interfaces Through Change Tracking, Annota- doi: 10.1109/TVCG.2009.111
tions, and Alternatives. In Proceedings of the SIGCHI Conference on [46] C. Neustaedter and P. Sengers. Autobiographical design in hci research:
Human Factors in Computing Systems, CHI ’10. ACM, 2010. doi: 10. designing and learning through use-it-yourself. In Proceedings of the

10
 To appear in IEEE Transactions on Visualization and Computer Graphics

SIGCHI Conference on Designing Interactive Systems, DIS ’12, pp. 514–

523. ACM, 2012. doi: 10.1145/2317956.2318034
[47] Observable, Inc. Observable. http://www.observablehq.com.
[48] S. Ohly, S. Sonnentag, C. Niessen, and D. Zapf. Diary studies in or-
ganizational research. Journal of Personnel Psychology, 2010. doi: 10.
1027/1866-5888/a000009
[49] F. K. Ozenc, M. Kim, J. Zimmerman, S. Oney, and B. Myers. How to Sup-
port Designers in Getting Hold of the Immaterial Material of Software. In
Proceedings of the SIGCHI Conference on Human Factors in Computing
Systems, CHI ’10. ACM, 2010. doi: 10.1145/1753326.1753707
[50] F. Perteneder, M. Bresler, E.-M. Grossauer, J. Leong, and M. Haller.
cLuster: Smart Clustering of Free-Hand Sketches on Large Interactive
Surfaces. In Proceedings of the ACM Symposium on User Interface
Software & Technology, UIST ’15. ACM, 2015. doi: 10.1145/2807442.
2807455
[51] D. Ren, M. Brehmer, B. Lee, T. Höllerer, and E. K. Choe. ChartAccent:
Annotation for data-driven storytelling. In 2017 IEEE Pacific Visualiza-
tion Symposium (PacificVis), Apr 2017. doi: 10.1109/PACIFICVIS.2017.
8031599
[52] M. Rettig. Prototyping for tiny fingers. Commun. ACM, 37(4):21–27, Apr.
1994. doi: 10.1145/175276.175288
[53] J. Rieman. The diary study: A workplace-oriented research tool to guide
laboratory efforts. In Proceedings of the INTERACT ’93 and CHI ’93
Conference on Human Factors in Computing Systems, CHI ’93, pp. 321–
326. ACM, 1993. doi: 10.1145/169059.169255
[54] A. Satyanarayan, D. Moritz, K. Wongsuphasawat, and J. Heer. Vega-Lite:
A Grammar of Interactive Graphics. IEEE Transactions on Visualization
and Computer Graphics, 23(1):341–350, Jan 2017. doi: 10.1109/tvcg.
2016.2599030
[55] M. Sedlmair, M. Meyer, and T. Munzner. Design Study Methodology:
Reflections from the Trenches and the Stacks. IEEE Transactions on
Visualization and Computer Graphics, 18(12), Dec 2012. doi: 10.1109/
TVCG.2012.213
[56] S. Sonnentag. Work, recovery activities, and individual well-being: A
diary study. Journal of occupational health psychology, 6(3):196, 2001.
doi: 10.1037/1076-8998.6.3.196
[57] C. Stolte and P. Hanrahan. Polaris: A system for query analysis and
visualization of multi-dimensional relational databases. In Proceedings of
the IEEE Symposium on Information Visualization (InfoVis). IEEE, 2000.
doi: 10.1109/infvis.2000.885086
[58] Tableau, Inc. Tableau Software. https://www.tableau.com.
[59] M. Tobiasz, P. Isenberg, and S. Carpendale. Lark: Coordinating Co-
located Collaboration with Information Visualization. IEEE Transactions
on Visualization and Computer Graphics, 15(6):1065–1072, Nov 2009.
doi: 10.1109/tvcg.2009.162
[60] A. Vande Moere and H. Purchase. On the role of design in information
visualization. Information Visualization, 10(4):356–371, 10 2011. doi: 10.
1177/1473871611415996
[61] B. Victor. Humane Representation of Thought: A Trail Map for the
21st Century. In Proceedings of the ACM Symposium on User Interface
Software and Technology, UIST ’14. ACM, 2014. doi: 10.1145/2642918.
2642920
[62] L. Wilkinson. The grammar of graphics. Wiley Interdisciplinary Reviews:
Computational Statistics, 2(6):673–677, Oct 2010. doi: 10.1002/wics.118
[63] J. Wood, A. Kachkaev, and J. Dykes. Design exposition with literate
visualization. IEEE Transactions on Visualization and Computer Graphics,
25(1):759–768, 2019. doi: 10.1109/TVCG.2018.2864836
[64] H. Xia, N. H. Riche, F. Chevalier, B. D. Araujo, and D. Wigdor. DataInk:
Direct and Creative Data-Oriented Drawing. In Proceedings of the SIGCHI
Conference on Human Factors in Computing Systems, CHI ’18, 2018.
[65] C. Yiu. Collaboration with distributed teams. Interactions, 21(4):50–53,
Jul 2014. doi: 10.1145/2627341

11