What Do We Talk About When We Talk About Dashboards?

Alper Sarikaya, Michael Correll, Lyn Bartram, Melanie Tory, and Danyel Fisher

Fig. 1: Klipfolio’s Social Media Manager Dashboard (DB065 from our example corpus, left) is a traditional dashboard, with
large numbers representing key metrics, and tiled graphs of real-time data. The UNCHR Refugees/Migrants Emergency Response
dashboard (DB117, right) also is a juxtaposition of key metrics and simple visualizations, but includes annotations and guided
narrative elements. Are both dashboards? Do design principles meant for one transfer to the other?

Abstract—Dashboards are one of the most common use cases for data visualization, and their design and contexts of use are
considerably different from exploratory visualization tools. In this paper, we look at the broad scope of how dashboards are used
in practice through an analysis of dashboard examples and documentation about their use. We systematically review the literature
surrounding dashboard use, construct a design space for dashboards, and identify major dashboard types. We characterize dashboards
by their design goals, levels of interaction, and the practices around them. Our framework and literature review suggest a number of
fruitful research directions to better support dashboard design, implementation, and use.
Index Terms—Dashboards, literature review, survey, design space, open coding

1 I NTRODUCTION
Visualization dashboards are ubiquitous. They are built and employed gle screen reports. Uniquely, compared to visualization modalities for
by nearly every industry, non-profit, and service organization to support presentation and exploration, dashboards bring together challenges of
data-driven decision making. They are used by students to track learn- at-a-glance reading, coordinated views, tracking data and both private
ing, and by individuals to monitor energy consumption and personal and shared awareness. Designers of dashboards must be mindful of
health. Despite their prevalence, the visualization research community literacy, contextually appropriate representations and visual language,
has rarely given dashboards their due consideration, with few excep- and social framing. We identify dashboards as a distinct area of visual-
tions [46]. Are dashboards simply an extension of known visualization ization that offers impactful directions for future research.
design principles? Or is there more to their design and use? We took a two-pronged approach to understanding practices around
We argue that dashboards are worthy of discussion and research in dashboard design and use. We conducted an exploratory survey of
their own right. Their ubiquity alone makes them worthy of study, as dashboards “in-the-wild” with the goal of discovering and identify-
the potential for impact is vast. But beyond that, they are interesting. ing different types of dashboard design. In parallel, we conducted
Dashboards are diverse, appearing in many different contexts. They are a multi-domain literature review in order to understand the practices
shifting and democratizing and diversifying as their use proliferates; surrounding dashboard use. The domain review allowed us to build a
their contexts of use are expanding beyond simple monitoring and sin- characterization of uses and domains of dashboards and identify issues
that the literature sees as urgent. These two complementary approaches
mutually informed each other and allowed us to see the breadth of the
• Alper Sarikaya is with Microsoft Corporation. E-mail: ill-defined space of dashboards.
[email protected]. We contribute a design space for dashboards that goes beyond princi-
• Michael Correll and Melanie Tory are with Tableau Research. E-mail: ples of visual encoding to include design dimensions such as functional
{mcorrell,mtory}@tableau.com. design, purpose, audience, and data semantics. We identify diverse
• Lyn Bartram is with Simon Fraser University. E-mail: [email protected]. categories of dashboards with unique sets of characteristics across the
• Danyel Fisher is with Honeycomb.io. This work was carried out while he dimensions. We also report issues and challenges surrounding dash-
was at Microsoft Research. E-mail: [email protected]. board use in practice, some of which emphasize the social context of
Manuscript received xx xxx. 201x; accepted xx xxx. 201x. Date of Publication dashboards as a primary interface to “big data.” Ultimately, we identify
xx xxx. 201x; date of current version xx xxx. 201x. For information on a set of interesting and critical research opportunities. We hope that our
obtaining reprints of this article, please send e-mail to: [email protected]. work will inspire and engage the community to embrace dashboards,
Digital Object Identifier: xx.xxxx/TVCG.201x.xxxxxxx study the challenges surrounding their use, and develop innovative
dashboard technologies with broad-reaching impact.
2 W HAT I S A DASHBOARD ? terms as we observed new design variations. We limited our codes
Even the definition of a dashboard is in flux. Few [19] describes dash- to facets of the dashboard that could be revealed through superficial
boards narrowly: “a predominantly visual information display that inspection of the representative images. This precluded analysis of other
people use to rapidly monitor current conditions that require a timely components of dashboard design, such as the types of computation or
response to fulfill a specific role.” This definition entails single-paged, the internal architecture. While these components are important to
glance-able views of updating data. Wexler et al. [61] offer a broader dashboard design, we chose instead to focus on codes which would
definition: “a visual display of data used to monitor conditions and/or allow categorization of all samples (even those for which we had very
facilitate understanding,” which can include infographic elements or little information) and which would highlight key design differences
narrative visualizations (such as Figure 1 right). Through both the between visual and functional dashboard genres.
domain review and the dashboard design survey, it became clear that Towards the end of our open-coding process, two of the authors
the term dashboard is widely used to refer to many different sorts of en- independently coded all of the dashboards using a preliminary set of
tities, challenging the dashboard stereotype familiar to the visualization design dimensions. They met over three sessions to review the evolving
community. Ubiquitous data, and visualization technologies avail- coding scheme and arrive at a mutual understanding with sufficient op-
able to the public, has broadened dashboard adoption to new domains. erationalization. They then completed coding the corpus of dashboard
Consequently, the dashboard concept has evolved from single-view examples and checked how closely their coding matched. At this point,
reporting screens to include interactive interfaces with multiple views they computed inter-rater reliability using Cohen’s kappa (κ = 0.64;
and purposes, including communication, learning, and motivation, in 86.5% agreement) in order to assess the reliability and repeatability of
addition to the classic notions of monitoring and decision support. the coding schema. They then discussed all the mismatches until they
Broadly, then, we identify two different major design perspectives. came to an agreement, revising the categories, codes, and definitions as
We distinguish between the visual genre of dashboards (a visual data needed to reach 100% consensus.
representation structured as a tiled layout of simple charts and/or large After deriving the design space categories and codes, we used these
numbers as in Figure 1 left) and the functional genre (an interactive factors to identify clusters in our corpus that highlight differences
display that enables real-time monitoring of dynamically updating between the dashboards we encountered, as well as functional and
data). While many data displays use the familiar “dashboard” visual visual commonalities. The resulting clusters that emerge from these
appearance, we found many tools that support the same functions but results can be found in Table 1, and are marked by colored numbers ( 1
have very different visual designs, especially dashboards meant for – 7 ). These diverse clusters reinforce the existence of ongoing shifts in
mobile devices. We do not attempt to provide a single authoritative how dashboards are conceptualized and designed, and point towards
definition of dashboards. Instead, we acknowledge a tension between areas in need of additional research and guidance.
the visual and functional genres. For the purposes of this survey, we We organize the 15 distinguishing factors into categories: purpose,
aim for inclusivity and consider a display a dashboard if it matches audience, visual & interactive features, and data semantics. We describe
either the visual genre or the functional genre, or both. The next two these aspects of dashboards in the following four sections.
sections explore our understanding of the dashboard space, derived
through our design survey and domain review. 3.2 Purpose
The intended use of a dashboard drives the choices in its visual design
3 DASHBOARD D ESIGN S URVEY and functional affordances. The factors presented here capture the roles
Using an exploratory methodology, we derived a design space consist- of each dashboard in the process of analysis and communication. We
ing of 15 visual and functional aspects of dashboards. find that the purpose of a dashboard has been substantially expanded
from the colloquial “operational” dashboard to capture decision-making
3.1 Survey Methodology at higher levels, and may not even support decision-making at all.
Our design space is based on a chosen corpus of 83 dashboards. We Decision Support (Strategic, Tactical, Operational): The decision
qualitatively analyzed and coded this collection of dashboards to derive support dimension reflects on the sorts of actionable decisions that
an initial design space. The corpus of images and sources is available dashboards are designed to support. Dashboards may be created to help
in the supplemental material. an organization choose and evaluate a strategy (e.g., “we want users
Each author collected his or her own set of visual displays that from around the world to be able to buy from our website”), refine their
could qualify as dashboards, with the intent to capture breadth of both tactics (e.g., “our CDN helps us keep the website globally available”), or
domains and visual design. The resulting broad sample was intention- evaluate their operations (e.g., “users in Seattle are seeing slow network
ally eclectic, to capture the current state of dashboards in the wild. response”). We chose this coding based on existing literature [12, 18],
We sourced example dashboards from (1) Tableau Public’s “Featured and note that these levels of decision support are not exclusive. Three
Dashboards,” (2) documentation from tools advertising “dashboarding” exemplar dashboard examples are shown in Figure 2.
features, (3) displays advertised on Twitter and elsewhere as “dash- Though we have described these terms by example (and they are
boards,” (4) Google image search results for the terms “data dashboard” defined within prior business literature, §5), we find it beneficial to
and “visual dashboard,” and (5) research papers in the domain review. think of the temporal lead and lag in the decision time. Operational
(For example, we chose to display the Strava user interface after noting dashboards describe the current and near past in terms of immediately
that Strava refers to their interface as a “dashboard.”) Our corpus of quantifiable metrics that can be tied to their responsible entities. For
dashboard examples evolved throughout the analysis. During our anal- example, if a management dashboard shows a gauge visual marking a
ysis, we realized that our initial corpus lacked representation of typical warning value, immediate action can be taken to rectify the problem (see
“business” dashboards, likely because these are usually confidential. Figure 2c). Strategic dashboards take a longer view on actionability—
We therefore intentionally sought documented examples of business combining many high-level metrics to drive decision-making over
dashboards to add to our collection. Additionally, we realized that we a longer temporal scale (see Figure 2a). We generally found that
needed specific dashboard examples; for tools and multi-dashboard these categories were not mutually exclusive—in these situations, the
collections, we chose a specific example, or removed them if no such dashboard tended to mimic an overview+detail design.
example was available. Our final coding scheme focused on the visual
design alone: we therefore excluded several kits or frameworks where Communication and Learning: We encountered several examples
no small set of representative images could be collected. of dashboards that did not solicit decision-making on any temporal
The dimensions in our design space were developed through an scale. The communication and learning factor identifies dashboards
iterative process that involved collaboratively open coding and sorting that visually appear to be dashboards but may not function as a tra-
the dashboards themselves, as well as reviewing the literature. In our ditional dashboard. Rather than eliciting decisions on the part of the
open coding process, we generated terms and categories that could viewer or analyst, these dashboards exist to communicate or educate the
describe the variation within our corpus of dashboards, adding new reader, who may lack the context surrounding the presented data. These
 (a) Strategic Dashboard (DB001) (b) Tactical Dashboard (DB106)

(c) Operational Dashboard (DB102) (d) Social Dashboard (DB028)

Fig. 2: Four dashboard exemplars demonstrating different attributes of dashboard design. A representative strategic dashboard (Fig. 2a)
emphasizes the trends of paying subscribers along with monthly breakdowns for increases and decreases. Fig. 2b is a tactical dashboard that uses
multiple metrics to summarize a student’s performance in a class. The operational dashboard (Fig. 2c) shows performance metrics that may
be actionable, but with no collective summarization. The social dashboard (Fig. 2d) uses social and personal data to situate the context of the
personal workout data. We demonstrate common factors of designs in the survey and highlight relevant challenges through our literature review.

dashboards echo an emerging trend of extending the functionality of 2 shows an individual’s home energy consumption. Another social
dashboards and their contexts of use (§5.2.1). dashboard is shown in Figure 2d, where social data is used to situate
personal fitness information.
3.3 Audience Required Visualization Literacy (Low, Medium, High): The com-
The visual and functional aspects of a dashboard typically reflect the plexity of visualizations on a dashboard can limit its comprehensibility.
intended audience, their domain and visualization experience, and their Instead of quantifying individual visual elements, we capture the visu-
agency relationship with the data. alization complexity through a quantized proxy of visualization literacy.
For the purposes of operationalization, we consider low literacy to
Circulation (Public, Social, Organizational, Individual): To under- capture basic visualization types such as bar and line charts with typical
stand the interpersonal circulation of a dashboard, we quantize the faceting and aggregation (e.g. Figure 4, DB101). Medium literacy
generalizability of audience into four groups, each becoming more adds visualization features such as combined dual axes, scatterplots,
specific and requiring more context (where the necessary context might cumulative measures, and heatmaps (e.g., DB005). We reserve the
not be included in the dashboard itself). A public dashboard is intended high literacy codes to capture those techniques known by a typical
for general consumption, and may describe societially-relevant data. visualization student or practitioner: radar, treemap, and network vi-
Dashboards for organizations are broadly applicable for many different sualizations, error bars or intervals, connected scatterplots, or other
individuals within a organizational construct, such that these viewers custom visualizations. For instance, DB052 contains an unfamiliar
share a common goal (say, supporting a business’ viability). Social radial view.
circulation captures contexts in which an individual controls the access
to the dashboard to individuals of their choosing, identifying scenarios Requires Advanced Domain Expertise: Many dashboards deal with
of sensitive data or analysis. Individual circulation captures dashboards business data or data general enough to be understood by a general
that quantify the individual and are generally not shared, except with audience, such as home electricity usage or personal finance. However,
trusted individuals (e.g., a doctor or financial planner). In coding, we some dashboards require domain expertise to understand, such as the
considered the largest potential group supported by the dashboard, and water metering metrics in Figure 4, DB034. This factor identifies cases
the potential values are therefore mutually exclusive. in which additional domain or analysis context is needed in order to
Some examples can be seen in Figure 4. The representative example understand the dashboard and take action.
for cluster 6 shows crime trends by state and type, presented for
the general public. Cluster 1 demonstrates a dashboard exploring 3.4 Visual Features & Interactivity
the customer relationship with the business, for use by individuals in While interactivity is a familiar aspect of visualization, we found sub-
an organization. An example of a social dashboard is shown in the stantial differences between different dashboards. Interaction can hap-
example for cluster 7 , presenting the value of different players for pen at a number of different places in the dashboard lifecycle. We
fantasy football. For an individual, the dashboard representing cluster distinguish between three types of interactivity: tools may allow a user
to design (or customize) the dashboard; they may allow faceting of
the data through data filters and slicers; and they may allow modify-
ing the state of the data and world based on the data presented within 1
the dashboard. These features speak to both the visual and functional
affordances of dashboards. 2
Construction and Composition: Many dashboards allow consumers
to modify the construction and composition of views. These dashboards 3
provide flexibility for the viewer to customize the placement of views,
modify the visual representations inside those views, or select the
particular dimensions and measures to visualize. For example, the
4
fitness dashboard in Figure 2d allows the user to choose which metrics
to visualize. We considered a dashboard to have these capabilities if 7 5
it contained a gear or × icons within each view tile (indicating the
functionality to remove or change properties of individual views), or if
individual views had evidence of being flexible in their placement (e.g.,
draggable corners). 6
Multipage: While dashboards have traditionally been all-in-one view
documents, some dashboards support tabbed layouts. These dashboards
allow viewers to switch between pages, which may have visualizations Fig. 3: Hierarchical clustering of the sample of 83 dashboards using a
that relate to a different component of decision-making or help to Hamming distance. We identified 7 clusters that exemplified different
provide necessary context. With multipage dashboards, however, it can functional and visual characteristics of dashboard design. An interactive
be difficult to consume the full breadth of information presented. version of this figure is available in the supplemental material.
Interactive Interface: Many dashboards support multiple coordinated
views. Interaction between views may involve faceting the data with
slicers and filters, cross-highlighting by selecting data items within the updatable dashboard accommodates changing data. While we antic-
views, and drilling up and down the levels of data hierarchy. These ipated many dashboards to fit this qualification, we identified in our
dashboards allow the consumer to focus their analysis on the data items early open-coding exercise several examples that were non-updatable:
that are relevant to them. We considered a dashboard to have interactive they described historical data or were highly customized to a singular
capabilities if we saw evidence of cross-highlighting (e.g., dimmed point in time. One such non-updatable dashboard is “Word Usage in
visual elements) or the presence of typical interactive components (e.g., Sacred Texts” (DB010 in supplemental material).
drop-down menus, slicers).
3.6 Factors We Could Not Capture in the Survey
Highlighting & Annotating: Several dashboards allow users to high-
Our open-coding exercise also identified several other factors that we
light and annotate particular views, thereby making persistent changes
were unable to operationalize for the purposes of coding dashboards:
to the dashboard. These changes do not apply to the underlying data;
analysis tasks, aspects of visual design, and context of use. However,
rather, they allow users to annotate displays for future examination or
we explore some of these issues in the domain review (section 5) and
for collaboration. For the purposes of operationalization, we consider
expand on potential avenues for future research in the discussion.
any dashboard that highlights or otherwise distinguishes a subset of
marks within any one view to support highlighting. 4 A PPLYING O UR D ESIGN S PACE
Modify Data or the World: Some dashboards have aspects of control We encoded each dashboard in our collection as a string, with each
panels: they have the ability to write back to the underlying database, encoded design dimension represented by a character. This allowed
or to control the external state of the world. Dashboards that support us to calculate a distance between dashboards as Hamming distance
“what-if” analysis, modeling and data entry can be examples of writing between strings. Figure 3 shows a minimized figure of hierarchical
back to the data source (or to a model). Other dashboards can interface clustering of our collection. Using these clusters, we identified seven
to processes outside of the data source, such as a smart home dashboard distinct clusters of dashboards, shown in Table 1. These seven clusters
that allows the viewer to turn off lights or adjust the thermostat. varied over the types of decisions they supported, along with the visual
and functional features afforded by each design. For the purposes
3.5 Additional Data Semantics of analysis and discussion, we group them by similar characteristics.
Other than visual and functional features, dashboards can provide Figure 4 shows an exemplar dashboard for each identified cluster.
valuable semantics about the data and processes that they visualize. We
capture these different type of semantics and the actions they elicit. Dashboards for Decision-Making: We found two distinct clusters for
decision-making (clusters 1 and 5 ). These dashboards tended to sup-
Alerting+Notification: A classic use of dashboards is to identify port either strategic (cluster 1 ) or operational decision-making (cluster
anomalies and highlight them for awareness and alerting purposes. 5 ). Many of these dashboards targeted audiences at the organizational
These dashboards maintain real-time connections to data, and use user- level, and afforded functionality that allowed the consumers to interact
or model-defined thresholds to raise an explicit alert to the viewer. with the views in order to focus on the data relevant for them. Over 3⁄4 of
These notifications indicate warning and critical scenarios, and prompt these dashboards contained some sort of benchmarks, allowing viewers
the viewer to take an immediate action to recitfy the issue. to identify areas of concern or improvement. Many of the examples in
Benchmarks: Benchmarks add indications of breaking user- or model- this cluster represent typical dashboards used daily in business contexts
defined thresholds, providing the viewer with additional data context. to understand sales and other metrics at a real-time level (operational)
These benchmarks can take the form of gauges with ideal goals or or over small/medium time periods (strategic).
warning thresholds (e.g., colors on guages in Figure 2c), marks that Static Dashboards for Awareness: We identified two clusters of static
show ideal directions (e.g., green up arrows and red down arrows), dashboards (no interactivity, single page) for slightly different contexts
status “lights” that turn green, or goals explicitly marked as met, not
of use (clusters 3 and 4 ). These dashboards tended to lack interac-
met, or exceeded.
tive elements commonly found in other dashboards, and tended to be
Updatable: Many dashboards are connected to datasets that are reg- designed for general awareness. Cluster 3 captures many dashboard
ularly updated, and the data within them automatically refreshes. An examples to be used in an operational context, generally providing
 Purpose Audience Visual Features Data Semantics

Alerting+Notification
Modify Data/World
Domain Expertise

Construction

Highlighting

Benchmarks
Vis Literacy

Interactivity
Operational
# Examples

Updateable
Multipage
Audience
Learning
Strategic

Tactical
Goal Cluster
1 Strategic Decision-Making 16 Y Y - N O - - - Y N N Y - - Y
Decision-Making
5 Operational Decision-Making 14 N Y Y N O - - - Y N N Y - Y Y

3 Static Operational 10 N N Y N O L - - - N N N - Y Y
Awareness
4 Static Organizational 8 - - N N O M - N N N - N N - Y

Motivation 2 Quantified Self 7 N N Y N I H N N Y N - Y - - Y

7 Dashboards Evolved 15 - - - - P H - - - - - - - - Y

Table 1: The dominant characteristics observed for each cluster of coded dashboards. A factor is considered dominant if it occurs at least 50%
over the prior probability, otherwise it is marked with a dash (-). Y indicates present or supported, N entails the opposite. P identifies the general
public, O represents organizational audiences, while I indicates dashboards designed for individual consumption. L, M, and H indicate low,
medium, and high visualization literacy required to understand the dashboard, respectively.

real-time data from sensors and metrics with low visualization literacy Cities, Social Organizations, Health Management, and Personal Visual
(line and bar charts) without much interactivity. However, we generally Analytics. We note that while we did not directly involve users in this
observed these dashboards to require advanced domain knowledge— research, many of the papers reported extensive user studies, often
additional context is needed to understand the semantic meaning of longitudinal investigations.
data trends and distributions. We examined literature primarily outside the fields of visualization
Cluster 4 captures those dashboards geared toward an organiza- and HCI, focusing on papers that described studies of real-world expe-
tional audience with generally no interactivity. These clusters are riences of using dashboards in multiple sectors. Papers in our survey
comprised of dashboard examples that would be shown in a static set- were sourced via Google Scholar and library searches with keywords
ting and could be consumed at-a-glance, such as those e-mailed to including dashboard, visualization, analytics, and monitoring. Our
executives and displayed on displays in a work environment. These two primary goal with the domain review was to identify challenges related
clusters seem to exemplify the design patterns of traditional dashboards. to dashboard technology and deployment. This review also informed
our design space and coding terms by identifying factors that dash-
Dashboards for Motivation and Learning: Outside of organizational board designers and users consider important in practice. These factors
contexts, we identified two clusters of dashboards concentrating on informed some of the coding terms. Most notably, “strategic”, “tacti-
individuals and the general public (clusters 2 and 6 , respectively). cal”, and “operational” are common purposes of dashboards used for
For dashboards tailored toward the individual, we observed primarily decision-making in the business literature [12, 18].
tactical and operation decision-making, with interactive interfaces and
alerting. These examples tended to exemplify dashboards in personal
5.1 Domains and Uses
matters, such as finance, exercise, and dieting. We could consider this
cluster to be a result of the proliferation of personal visual analytics. We commonly think of dashboards in business organizations, with goals
Dashboards designed to be used by the general public exhibited such as optimizing decision making, enhancing operational efficiency,
more ambiguous decision-making affordances. While about half ex- increasing data visibility, driving strategy and transparency, reducing
hibited strategic purposes, all seemed to be designed for the purposes costs, and facilitating communication [23, 36, 37, 65, 66]. Dashboards
of communication to and education of the consumer. In this cluster, are commonly categorized by the type of decision-making they support:
dashboards tended not to have alerting or benchmarks, opting instead to strategic, tactical or operational [36, 39]. Even within business intelli-
present the data plainly to allow the viewer to come to an independent gence there is considerable diversity in dashboard use. A BI dashboard
conclusion. We observed several examples of public health dashboards, is commonly now more than a single-view reporting screen: it is a
dashboards of crime rates, and other types of civic data here. portal to the information needed for some goal and may serve multiple
analytical tasks [12,36,65]. Yet it is outside BI that dashboard purposes
Dashboards Evolved: The last cluster, 7 , that we identified was a and needs become even more widely varied.
catch-all that did not fit into the other clusters. These examples tended As an example, health organizations have been adapting dashboards
to exemplify combinations of characteristics independent of the other at both large-scale (hospital management [60]) and patient-care lev-
clusters. Many of these examples visually appeared as dashboards, but els [7, 17, 63], with a primary goal of supporting collaboration and
may not fit the strictest definition of dashboard functionality. Figure 4 awareness across diverse roles, time-frames and expertise. Urban
shows an example of a dashboard-style visualization of football players informatics [9, 25, 34, 35, 40] and community organizations / non-
and their statistics for fantasy football. profits [8,27,47,48] all face challenges of integrating multiple disparate
data sources, serving a large and diverse set of stakeholders, adapt-
5 L ESSONS FROM THE F IELD : DASHBOARDS IN U SE ing to multiple platforms including mobile devices, and developing
In parallel with the dashboard survey, we conducted a multi-disciplinary metrics and representations to represent intangible outcomes such as
review of dashboards in practice by examining literature reporting community awareness, engagement, and trust. Both learning analytics
case studies, user studies, design exercises and deployment reports dashboards (e.g., [51, 57, 58, 64]) and personal behavior tracking dash-
of dashboards used in Business Intelligence (BI), Education, Smart boards (e.g., [21, 50]) may incorporate a social sharing and comparison
1 Strategic decision-making (DB052) 2 Quantified Self (DB021) 3 Static Operational (DB034) 4 Static Organizational (DB101)

5 Operational decision-making (DB036) 6 Communication (DB005) 7 Dashboards Evolved (DB045)

Fig. 4: Exemplar dashboards selected from our seven derived clusters. Clusters 1 and 5 demonstrate dashboards specifically targeting
decision-making, while clusters 3 and 4 target awareness on behalf of the consumer. Cluster 2 targets the somewhat novel quantified self
scenario (smart-home dashboard), while 6 represents dashboards tailored for general-purpose communication. Cluster 7 captures some novel
extensions to traditional dashboards.

aspect, raising representational challenges and concerns about privacy. what-if simulations [37, 39, 44, 49, 61, 64, 65], or engagement features
Across these diverse domains, dashboard limitations were reflected in such as goal setting and gamification [58]. Automatic classification
a number of common themes. Some of these match current research is one feature already showing benefits, such as automatically classi-
questions in the visualization community, while others are novel. We fying a user’s location (e.g., home, work) from GPS data in personal
discuss each in turn. monitoring dashboards [54].
An issue of communication is related to storytelling ability. Dash-
5.2 Familiar Challenges boards are increasingly used for decision making and communication
Like many visualization tools, dashboards share issues that are familiar across contexts: top-down, within departments, and across the orga-
to visualization research; these are exacerbated in dashboards due to nization [29, 36]. Dashboards that capture only the data and not the
their diverse contexts of use and user populations. Major challenges semantics of the data, or what was done in response to the data, can
related to the actual dashboard artefact are functional flexibility (in- be insufficient for communication purposes. In BI, people often take
cluding more control over authorship and configuration) and visual and screenshots of dashboards and put them into slide presentations in order
analytic literacy. to annotate them with contextual information [13, 14, 38], suggesting a
need for more powerful storytelling features.
5.2.1 End-User Flexibility
Numerous articles expressed a desire for greater flexibility in dash- 5.2.2 Visual, Analytic, and Data Literacy
boards, including drill down options, level of detail adjustments, search, A consistent theme in the literature was the challenge of supporting
comparison support, and adaptability to different users, situations, and users with weak visual, analytic, or data literacy [12, 27, 33, 38, 49].
display devices or multi-display environments [6,15,21,29,36,37,39,49, Galesic et al. [22] defined a metric of three levels of graph literacy:
51, 57, 61–63, 65]. While many interactive dashboarding tools provide finding and extracting data (Level 1); integrating data and finding
support for basic functions like drill-down and search, automatic adapta- relations (Level 2); and analyzing implicit relationships, generating,
tion to different users and display environments is still an open research and predicting (Level 3). Wakeling et al. [59] found that users were
problem. Moreover, many dashboarding tools offer little opportunity much more competent with Level 1 and 2 questions than Level 3, except
for end users to reconfigure and customize views, a feature that was with bar charts where they could answer Level 3 questions. A frequent
reported important in BI [29,36,65], health management [15,63], learn- example of analytic illiteracy was the confusion between correlation
ing analytics [51, 58], and personal applications [21, 50]. This includes and causation. Visual literacy also included a lack of understanding of
the ability to integrate hybrid data sets [60] and add annotations to the metrics and representation conventions across stakeholder groups [3].
data. In a recent project of dashboards for citizen engagement, users Ease of dashboard use was repeatedly reported as a problem. There is
wanted to enter their own data, configure their own views, and redefine a burden on authors to train end users, either through explicit instruction
critical metrics to evaluate new proposals [30]. (e.g. tutorials) or guidance built into the dashboard itself [12, 13, 38].
As the scope of dashboard use moves from merely reporting perfor- While supporting non-expert users has been a theme in the visualization
mance via proxy metrics to more in-depth problem solving [1,11], users research literature for some time, the extent of the problem was beyond
also want more analytical support in their tools, particularly “smart” fea- our expectations: even seemingly ‘simple’ interactive features such as
tures such as automated analytics (pattern identification, prediction) and filters were often considered too complex for casual users [12].
 Dashboard authors went to great lengths to support non-experts. Adaptivity: Different tasks require different levels of data: both richer
They frequently described how they reduced interactive functionality access to data through drill-down or roll-up and task-appropriate data
or customizability to reduce complexity [2, 12, 38]. As stated by Ma- filtering [7, 16, 23, 29, 37, 63, 65]. Studies show that tasks with higher
lik [38], “...most users do not have the time or motivation to learn new uncertainty require more disaggregated data but showing more detail
software applications. The golden rule is to simplify, simplify, simplify!” all the time leads to information overload and decision inaccuracy [4].
Similarly [12], “If designed properly, tactical dashboards can meet For dashboards used in critical applications (e.g. healthcare), timely
the needs of both casual and power users. The key is to... support access to the “right data” is particularly important as decisions are
the information needs of power users and then turn off functionality made collaboratively across different situational and personnel contexts
as needed to avoid overwhelming casual users with too many bells [17, 63]. For personal analytics applications, users tend to want control
and whistles or pathways to explore.” Another strategy is to establish to select the data that matters for their personal goals [21, 62].
cross-organization design principles, “The finance manager should be
able to easily drive the credit or treasury dashboard because it looks Data and metadata: Many cases we read discussed the need for the
and behaves similarly to the finance dashboard.” [38] provision of metadata. A number of these issues relate to data quality:
A similar learning curve faces dashboard authors themselves [13]. completeness [27, 47, 56], provenance [9], accountability [8, 31, 48, 64]
With the explosion of data democratization, dashboards are being built and uncertainty [47]. These concerns were especially prevalent in the
by people with a vast diversity of skills and knowledge, most of whom urban informatics and social organization domains, where quality of
are not data analytics or visual design experts. Novice authors rely data, trust and accountability are critical [24, 48] but are often poorly
heavily on experts, templates, and learning resources to learn both dash- elicited by current dashboards [31, 35]. Explicitly referencing where
boarding tools as well as visualization design principles and practices. the data come from, and the degree of certainty in that data, are seen as
Moreover, dashboard authors are often uncertain how to evaluate the important to enhancing public trust [9, 41, 47]. These issues are more
effectiveness of the dashboards that they build [25, 58]. These obser- complex when data are derived from automated algorithms and the
vations emphasize the need for accessible templates, guidelines, and reasoning is “invisible”. Researchers in policy note that providing infor-
learning resources to help non-experts become proficient. mation and explanation about the underlying data and the algorithmic
and statistical techniques performed on it is important for both account-
ability and trust [9, 10, 64]. A different dimension of data and analytic
5.3 Dashboards Emphasize New Challenges
“depth” emerged in reports of interactive dashboards with additional
Some of the challenges regarding dashboards have a new twist, which data views: indicating the level of analytic functionality [36, 37] or data
indicate new opportunities for visualization research. These include detail [24] behind the view.
choices of data and representation that are critical to dashboard contexts
of use; and understanding the broader social impact of dashboards. 5.3.2 Social Impact
In our review, dashboards were often the primary interface to how
5.3.1 Data Design
people use “big data:” consequently, dashboard use is often a proxy for
Many issues related to data as they interface with dashboards emerged the social impacts of big data practices. Concerns raised included data-
repeatedly over different data domains: confusion around the choice limited thinking, unintended reactions to data in the workplace, privacy,
of metrics, impoverished vocabulary, the nature of adaptive views, and trust, and issues related to “data democratization:” who controls the
communicating metadata. data, who configures the view, and who gets to see what. These issues
widen the discussion of dashboard design, introducing challenges of
Metrics: Data choice and how they are presented affect both the scope representation and framing relevant to both designers and researchers.
and nature of decision-making [24, 29]. Metrics such as Key Perfor-
mance Indicators (KPIs) communicate not only performance but also Data-driven thinking: Data views are not neutral although they con-
priorities and values. As an example, process KPIs matter for tracking vey the impression of objectivity and “truth” [24, 65]: how they are
compliance [43] (”Are we following all protocols?”) but are less appro- framed and visualized reflects the values and perspectives of the authors
priate for tracking performance relative to desired outcomes (”Is the and determines their interpretation [28]. Rall et al. [48] allude to the
client getting better?”) [7]. Performance dashboards often suffer from danger of incorrectly designed visualizations in human rights advo-
a poor taxonomy of strategic goals [29, 35, 65]; several studies suggest cacy: “visualized quantitative data lends human rights the stature of
explicitly visualizing strategy maps that link KPIs specific to subgroups science.” Similarly, in a study of dashboards for school health, people
to the organization’s larger strategic objectives [36, 65]. Choosing the interpreted the data as “trustworthy and definitive” [10] because the
right metrics poses challenges for those less versed in analytics culture, visual organization emphasized those associations as objective.
particularly where trustworthy communication is the primary purpose Dashboard use often restricts the scope of interpretation and decision
such as in social organizations [8, 48] and public outreach. Most social making [3, 24, 31, 64]. Angrave calls dashboards the “visual expression
organizations lack both organizational and data literacy to build up of the KPI mindset” [3] and is one of many authors who caution that hu-
metrics and representations [8, 27, 48]. man behaviour can often not be simply quantified [23,29,32]. Education
studies highlight challenges of lack of accountability in algorithmically
Impoverished data vocabulary: Our review also identified the need generated prediction and concerns that these approaches are displacing
for more sophisticated metrics and representations to express the nu- the pedagogic expertise of educators [10, 42, 64]. Similarly, urban stud-
ances of complex problem solving [3, 24, 31, 64, 65]. The California ies papers identified several visualization challenges: concepts like the
School Dashboard uses traffic-light colour coding designed to sim- flow of information in a society, contextual information associated with
plify data interpretation for parents. It caused concern among educa- places, and the emergence of communities [9,30,34,35,40]. These ques-
tors and parents around over-simplification of school “health” [55]. tions of knowledge extraction are critical for both assessing community
Such constrained “scorecard” views and quantitative models amenable health [10, 40, 45] and for supporting citizen engagement [30, 34].
to traditional data analytics can impose mechanistic, unsophisticated
decision-making regimes [3, 24, 29, 35, 64] that are susceptible to mis- Social data as context: Data about peer performance or status are
interpretation. often used in dashboards to promote engagement and motivation (e.g.,
Data choice re-frames problem definition and thus implicitly refo- in personal dashboards [21, 26, 50], learning analytics [51] or em-
cuses the values of possible solutions [3, 24, 29, 31, 45]. O’Connell et ployee/department comparisons [3]). How these data are framed and
al. [45] reported a project to engage people in a dialogue around pub- represented present challenges. Comparisons that emphasize competi-
lic safety using a community dashboard. Where urban social planners tion can be highly motivational. For example, “the overriding element
previously used an asset-based model of neighborhood health (social co- that helped to persuade some to remain engaged was the ability to share
hesion, resilience), the move to a data-driven representation refocused and compare energy consumption” [21]. Yet such competition can be
the conversation on a deficit-based model (crime, code violations). de-motivating when the user feels the thresholds of performance are
out of reach [52]. Also, raw or simplistically aggregated data may not are concerns that crosscut nearly every dashboard-like juxtaposition
give the right picture (in one learning analytics study, students wanted of multiple visualizations. Yet, the differing purposes across the sub-
“more equitable” representations of comparison with their peers rather categories suggest that there are design principles that do not transfer.
than aggregate percentages [51]). An author of a static dashboard for an organization (cluster 4 ) may
wish to eschew customization, to ensure that the entire organization is
Sharing, security, and trust: Dashboards are increasingly used in
making decisions based on a complete and universal picture of the data.
a social context. Their use now crosses hierarchical and departmen-
However, the designer of a dashboard for personal informatics (cluster
tal boundaries, highlighting the need to integrate and align both key
metrics and the representation and framing of them to support collab- 2 ) would likely wish to afford customization by a user who would
oration and communication [36]. When dashboards become portals want to include the most relevant metrics for their lived experience.
to the information system, there are questions of access and agency. As the functional use of dashboards is shifting and expanding, ex-
Who can see the data at all levels? Is access managed by role [29, 36]? isting design considerations may too be expanding: in the case where
Authoring models for data can be both unclear and insecure [30, 45], dashboards are encroaching into information presentation territories
leading to issues of trust. A case study of a successful dashboard de- previously considered to be the realm of infographics or narrative visu-
ployment in hospital management [60] identified two key aspects to alization, designers of dashboards must become mindful of storytelling,
promote shared understanding and employee acceptance: shared con- rhetoric, and persuasion, a point we explore further in Section 6.2.
text (dashboards included details for the specific area and an overview Our “Dashboards Evolved” cluster ( 7 ) captures an emerging space
of organizational strategic objectives / targets) and transparency (90 of dashboards that do not fit neatly into our prior conceptions of what
percent of dashboards could be seen by every employee). dashboards look like or how they function. Just as other genres in areas
like film and literature change over time as exemplars circulate and
Privacy: How individual data are presented impact both privacy and tastes and fashions evolve, dashboards too may undergo radical shifts
motivation. Studies show that students want control over what others in their conceptualization and visual signatures.
can see and when, especially when they are viewing the data in public
or mobile situations [51]. Beyond explicitly social data, many articles 6.2 Where Do we Go From Here? Research Challenges
reported a dysfunctional impact of dashboards related to concerns of We observed major design challenges of functional flexibility, visual
surveillance and anxiety about what data were being captured, who had and data literacy expectations, data expressiveness and framing, and
access, and how they were interpreted [10, 17, 29, 51, 65]. social impact. But what can dashboard designers and researchers do to
address these questions?
6 D ISCUSSION We noted the importance of enhanced functional and visual design
Our exploration of dashboards took two complementary lenses: a in our reviews. Our open-coding exercise identified several factors that
perspective focused on the artefact design (the dashboards survey) and we were unable to operationalize for the purposes of coding dashboards,
one focused on artefact practices (the domain review). They highlighted notably the complex analysis tasks identified in our field survey. Visu-
design issues and challenges that dashboard designers should consider, alization research commonly uses analysis task as an abstract proxy to
many of which pose interesting research questions for the visualization compare the efficacy of different designs. More detailed usage analysis
community. The clearest implication of our work is that we need to would expose the combination and composition of tasks (such is as
stop thinking of dashboards as a single entity and actually explore and used by Brehmer and Munzner [5] or Schulz et al. [53]). Identifying
experiment with design separately for different purposes, contexts of how dashboards help users analyze their content from a visualization
use, and data environments. perspective is important future work, particularly for non-experts.
We need better ways to assess and support visualization and analytic
6.1 The Shifting Design Space of Dashboards literacy with dashboard design. Enabling access to more data and pro-
Our domain research and the clusters in the design space directly viding more extensive analytical support can both extend the utility of
address the tension we identify between dashboards as a visual genre a professional dashboard and can also scaffold and guide users who are
(e.g., a tiled layout of key metrics visualized in simple charts) and less versed in data and visualization, enabling them to ask and answer
dashboards as a functional tool (e.g., affording real-time monitoring of their own questions. The challenge of designing for users with different
operations). They confirm that the term “dashboard” does not pick out levels of visual and analytic literacy cannot be addressed by assuming
a unique method of organizing, presenting, and using data, but rather fixed designs for singular audiences. Dashboards are increasingly used
covers a diverse set of practices. for sharing information across roles, domain expertise and motivations
The fact that widely differing objects all situate themselves as dash- both within and across organizations. They provide a common frame
boards complicates their design, as there is no guarantee that the design of reference with shared data and tailored representations for specific
principles from one category of dashboard neatly transfer to another. users. The current constructs we have for visual literacy [7,63] fall short
For instance, layout decisions that are effective for the glance-ability of capturing the diverse purposes and contexts of use that influence
of key metrics in operational decision-making dashboards (cluster 5 ) analytic success and may or may not facilitate proficiency gains. We
may not be effective for the narrative and anchoring needs of commu- need new methods to evaluate these complex comprehension tasks.
Future work could also extend our framework with dimensions vi-
nication dashboards (cluster 6 ). A designer of such a commuication
sual design, which we left out for tractability. Aspects such as layout
dashboard ( 6 ) may wish to preserve the temporal or logical ordering and arrangements can have significant impact on the efficacy of a dash-
of views in order to clearly communicate an idea, whereas a designer board, especially in relation to the analytic process of the consumer.
of a operational dashboard ( 5 ) may wish to order views such that the In a context of a dashboard that has a potentially wide variety of uses,
most critical information is the most prominent, no matter the tem- other aspects of design may become important to consider, such as
poral or semantic relationship between views. Designers of strategic affect [6] or memorability [5]. As dashboards venture into informa-
decision-making dashboards (cluster 1 ) may need to consider issues tion presentation territories previously considered to be the realm of
of supporting interactive temporal aggregation, levels of detail, and infographics or narrative visualization, designers of dashboards must
re-analysis—concerns that would not arise in static, short-term, static become mindful of storytelling, rhetoric, and persuasion.
operational dashboards (cluster 3 ). These differences reflect the grow- We suggest that the most compelling and difficult challenges for
ing needs for increased flexibility and extended functionality identified dashboard designers (and by extension, visualization designers) re-
in the domain research. late to our impoverished data and representational vocabulary. The
We contend there are design principles that are shared across a broad popularity of infographic elements and styles notable in the clusters
category of artifacts in the visual genre of dashboards. Consistency suggests an effort to capture elements of the dashboard purpose and
among views [46], the signaling of dashboard affordances [61], and a background that are not easily reduced to data attribute-visual feature
general concern for managing complexity and comprehensibility [18], mappings. Beyond this, we see an expressed demand for visualization
that captures qualitative and more intangible concepts (e.g. organiza- 7 L IMITATIONS
tion health [34], social quality [13], and context [27, 44]). Mitigating
Our work is subject to some caveats. Most importantly, our method-
the reductive impact of data-limited thinking models will require a
ology explored dashboards and their use indirectly through examples
richer and more expressive data and visual lexicon. This argues for a
and literature. We did not directly consult dashboard users or designers
more comprehensive study of the affective and cognitive affordances
and may therefore have missed some important design considerations,
of these new forms. This applies not only to the data themselves but
challenges, and potential mismatches between intended and actual sys-
also to the overall framing and contextual information of the dashboard.
tem use. While the insights revealed by the literature do correspond
Provenance, relation to the larger context of shared goals, uncertainty,
to anecdotal reports we have heard about dashboard use, we consider
explanation of algorithmic conclusions [12, 40, 68] and appropriate
systematic studies involving actual dashboard designers and consumers
social context (comparison, discussion) are examples of metadata and
as an especially important avenue for future research.
framing that may alleviate the tendency to over-simplify problem think-
ing and reduce concerns of accountability and trust. These in turn In addition, the dashboards examined in our survey were intention-
introduce challenges of representation and visual literacy. ally diverse but are by no means a representative sample. It is encour-
aging that our resulting categories are semantically meaningful and can
capture the commonalities within our diverse collection. Nonetheless,
6.3 Comparison to Existing Frameworks we fully anticipate that examining additional dashboards, use cases,
and domains will reveal new insights and additional categories. Addi-
One of our goals in this work was to gain a comprehensive under-
tionally, many of our design dimensions and observations may apply
standing of the dashboard design space and to begin to develop a more
to other types of visual analytic systems; future work could explore
coherent framework of design principles. Our research confirms that
these synergies and attempt to more clearly deliniate the challenges
the term “dashboard” does not pick out a unique method of organizing,
unique to dashboards alone. Our design space should be taken as a first
presenting, and using data, but rather covers a diverse set of practices.
step rather than a complete characterization; we hope that this work
There is a lack of design frameworks that can accurately describe this
will inspire others to further develop and refine it through alternative
emerging space: current models address limited aspects, but, we pro-
methodologies and complementary examples.
pose, fail to capture its extent and complexity. For example, Few has
written extensively about dashboard design principles for strategic,
tactical, and operational decision-making [18]. He also distinguished 8 A C ALL TO ACTION
between what he calls “dashboards” (single screen, used only for moni- Dashboards might seem, at first glance, comparatively simple: a handful
toring) and “faceted analytical displays” (used for analysis) [20]. Our of core chart types, placed in multiple coordinated views, might seem to
clusters go further than this in distinguishing the diversity of dashboard pose few technical challenges. This surface simplicity can be deceiving:
types, as defined by their purposes and characteristics. Our descriptive the whole of dashboards are far more then the sum of their parts.
rather than prescriptive framework may fail to isolate best practices in Dashboards are both ubiquitous and critically important in a data-
dashboard design, but by widening our framework to extant practices, driven world. Uncountable numbers of businesses, non-profit organi-
we can identify areas in need of further research attention. zations, and community groups depend on dashboards every day to
There have also been several design frameworks proposed for tra- get their work done. For many people in these organizations, dash-
ditional business intelligence dashboards that address both visual and boards may be their first (or only) encounter with data. Moreover,
functional scope [36, 39, 65]. Marx et al. [39] identified three dimen- dashboard use has spilled out of organizational boundaries to individu-
sions to consider in BI dashboard design: a comprehensive information als and the general public. Everybody is using dashboards, but are they
model (scoped relevant to both task and role), functional breadth (e.g., well-prepared and supported?
drill-down, simulations, alerts, and mobile access), and flexibility to The ubiquity of dashboards presents an opportunity. Visualization
accommodate different levels of visual and data literacy. Lea and research that can facilitate dashboard design, development, and use
Nah [36] extended Marx et al.’s framework to a multi-layered model of has potential to change the lives of millions of people. Today, dash-
linked dashboards supporting the three types of organizational decision- board authors and consumers are making do with what they’ve got.
making, based on data, display type, and analytical functions. These It’s not enough. Dashboard designers struggle with one-size-fits-all
frameworks share some piecemeal elements with our own (e.g. pur- dashboarding tools that fail to reflect the diversity of user needs and
poses, functions, and visual literacy) but are much less comprehensive. goals. They struggle with challenges of visual and analytic literacy,
Most similar to our framework is one described by Yigitbasioglu et constrained data-driven thinking, and development of metrics that really
al. [65], who framed dashboard design as optimizing “fit” between 4 reflect the things they care about. They are asking for greater flexibility,
dimensions: purposes, users, design features (functional and visual) customization, adaptability, and deeper analytics.
and outcomes. Purposes included planning, communication, mon- Nearly every aspect of visualization and visual analytics research
itoring and analysis. Functional features included analytical tools, done in our community could be considered in the context of dash-
drill-down/roll-up, view reconfiguration, and notifications, many of boards. How does visual perception change in dashboards? What does
which are also present in our framework. Functional fit refers to how storytelling support look like for dashboards? How can automated
well a dashboard’s functions align with its purpose. Both functional analytical approaches be integrated? How can we support customizabil-
and visual features enable cognitive fit with different types of users. ity and personalization for casual users? Can a dashboard itself teach
These frameworks explicitly acknowledge emerging characteristics literacy skills? The list of potentially impactful directions is immense.
of dashboard requirements in business applications: an increasing diver- Are we up to the challenge?
sity in users (personality, task and role); dynamic flexibility in purpose As a research community, we can continue to see the dashboard in-
and function (i.e., serving more than one type of decision-making), and dustry grow in parallel to our own work. Or we can embrace dashboards
more sophisticated interaction with data (level of detail adjustment, as our own, engage with dashboard users in the wild, and tackle the
reconfiguring the displays, integrating analytical and what-if tools). interesting challenges they face. We invite the visualization community
However, they fail to encapsulate design dimensions relating to the to systematically study dashboard design, create the dashboard of the
myriad of ways in which dashboards are used, the ways they affect future, and in doing so, impact the people who rely on visualization
organizational practice and culture (social fit), and the challenges posed every day.
by the need for more flexible data fit (new metrics, richer representa-
tions). These issues are extremely relevant to the new generation of
ACKNOWLEDGMENTS
dashboards. Existing frameworks have also been explored solely in
the business domain; our broader survey revealed new aspects such as We thank Microsoft Research, Tableau Research, Simon Fraser Uni-
dashboards for learning, interactivity that modifies the outside world, versity, and NSERC for supporting cross-organization collaborative
and dimensions related to data semantics. research, and the anonymous reviewers for their input and critique.
R EFERENCES [23] J. Goldmeier and P. Duggirala. Dashboards for Excel. Sept. 2015.
[24] J. Gray, L. Bounegru, S. Milan, and P. Ciuccarelli. Ways of Seeing Data:
[1] A. Abbasi, S. Sarker, and R. Chiang. Big data research in information Toward a Critical Literacy for Data Visualizations as Research Objects and
systems: Toward an inclusive research agenda. Journal of the Association Research Devices. In Innovative Methods in Media and Communication
for Information Systems, 17(2):i–xxxii, Feb. 2016. Research, pp. 227–251. Springer International Publishing, Cham, Dec.
[2] C. Alonso-Fernandez, A. Calvo, M. Freire, I. Martinez-Ortiz, and 2016.
B. Fernandez-Manjon. Systematizing game learning analytics for se- [25] S. Gray, O. O’Brien, and S. Hügel. Collecting and Visualizing Real-Time
rious games. In Global Engineering Education Conference (EDUCON), Urban Data through City Dashboards. Built Environment, 42(3):498–509,
pp. 1111–1118. IEEE, 2017. Oct. 2016.
[3] D. Angrave, A. Charlwood, I. Kirkpatrick, M. Lawrence, and M. Stuart. [26] D. Huang, M. Tory, B. A. Aseniero, L. Bartram, S. Bateman, S. Carpendale,
HR and analytics: why HR is set to fail the big data challenge. Human A. Tang, and R. Woodbury. Personal visualization and personal visual
Resource Management Journal, 26(1):1–11, Jan. 2016. analytics. IEEE Transactions on Visualization and Computer Graphics,
[4] I. Benbasat and A. Dexter. Value and events approaches to accounting: an 21(3):420–433, 2015.
experimental evaluation. Accounting Review, 54(4):735–749, 1979. [27] B. Kanter and K. Delahaye Paine. Measuring the networked nonprofit
[5] M. Brehmer and T. Munzner. A Multi-Level Typology of Abstract Vi- using data to change the world. Jossey-Bass, San Francisco, 2012.
sualization Tasks. IEEE Transactions on Visualization and Computer [28] H. Kennedy, R. L. Hill, G. Aiello, and W. Allen. The work that visualisa-
Graphics, 19(12):2376–2385, 2013. doi: 10.1109/TVCG.2013.124 tion conventions do. Information, Communication & Society, 19(6):715–
[6] E. K. Choe, B. Lee, H. Zhu, N. H. Riche, and D. Baur. Understanding 735, Mar. 2016.
self-reflection: how people reflect on personal data through visual data [29] H. Kerzner. Project management metrics, KPIs, and dashboards. Wiley,
exploration. pp. 173–182. ACM Press, 2017. doi: 10.1145/3154862. Nov. 2017.
3154881 [30] Z. Khan, J. Dambruch, J. Peters-Anders, A. Sackl, A. Strasser, P. Frhlich,
[7] B. F. Chorpita, A. Bernstein, E. L. Daleiden, and The Research Network S. Templer, and K. Soomro. Developing Knowledge-Based Citizen Partici-
on Youth Mental Health. Driving with Roadmaps and Dashboards: Us- pation Platform to Support Smart City Decision Making: The Smarticipate
ing Information Resources to Structure the Decision Models in Service Case Study. Information, 8(2):47, Apr. 2017. doi: 10.3390/info8020047
Organizations. In Administration and Policy in Mental Health and Mental [31] R. Kitchin. Making sense of smart cities: addressing present shortcomings.
Health Services Research, pp. 114–123, Nov. 2007. In Cambridge Journal of Regions, Economy and Society, pp. 131–136,
[8] J. Coffman, T. Beer, P. Patrizi, and E. H. Thompson. Benchmarking Mar. 2015.
Evaluation in Foundations: Do We Know What We Are Doing? The [32] R. Kitchin. Urban data and city dashboards: Six key issues. SocArXiv,
Foundation Review, 5(2):36–51, July 2013. Sept. 2016. doi: 10.17605/OSF.IO/K2EPN
[9] A. Crooks, D. Pfoser, A. Jenkins, A. Croitoru, A. Stefanidis, D. Smith, [33] R. Kitchin, S. Maalsen, and G. McArdle. The praxis and politics of
S. Karagiorgou, A. Efentakis, and G. Lamprianidis. Crowdsourcing urban building urban dashboards. Geoforum, 77:93–101, Dec. 2016. doi: 10.
form and function. International Journal of Geographical Information 1016/j.geoforum.2016.10.006
Science, 29(5):720–741, Jan. 2015. [34] R. Kitchin and G. McArdle. Urban data and city dashboards: six key
[10] R. Crooks. Representationalism at work: dashboards and data analytics in issues. In Data and the City, pp. 1–21. Sept. 2016.
urban education. In Educational Media International, pp. 1–14. Routledge, [35] K. Kourtit and P. Nijkamp. Big data dashboards as smart decision support
Dec. 2017. tools for i-cities – An experiment on stockholm. In Land Use Policy, pp.
[11] M. Djerdjouri and A. Mehailia. Adapting business analytics to levelerage 24–35. Elsevier, Feb. 2018.
enterprise data assets. In R. Benlamri and M. Sparer, eds., Leadership, In- [36] B. R. Lea and F. H. N. Nah. Usability of Performance Dashboards,
novation and Entrepreneurship as Driving Forces of the Global Economy. Usefulness of Operational and Tactical Support, and Quality of Strategic
Nov. 2016. Support: A Research Framework. HIMI HCII 2013, Part II, LNCS 8017,
[12] W. W. Eckerson. Performance dashboards: measuring, monitoring, and pp. 116–123, June 2013.
managing your business. John Wiley & Sons, 2010. [37] H. Lempinen. Constructing a design framework for performance dash-
[13] M. Elias, M.-A. Aufaure, and A. Bezerianos. Storytelling in visual an- boards. In Nordic Contributions in IS Research, pp. 109–130. Springer
alytics tools for business intelligence. In IFIP Conference on Human- Berlin Heidelberg, 2012.
Computer Interaction, pp. 280–297. Springer, 2013. [38] S. Malik. Enterprise dashboards: design and best practices for IT. John
[14] M. Elias and A. Bezerianos. Annotating bi visualization dashboards: Wiley & Sons, 2005.
Needs & challenges. In Proceedings of the SIGCHI Conference on Human [39] F. Marx, J. H. Mayer, and R. Winter. Six principles for redesigning
Factors in Computing Systems, pp. 1641–1650. ACM, 2012. executive information systems—findings of a survey and evaluation of
[15] A. Faiola, P. Srinivas, and B. Doebbeling. A ubiquitous situation-aware a prototype. ACM Transactions on Management Information Systems,
data visualization dashboard to reduce ICU clinician cognitive load. In 2(4):1–19, Dec. 2011.
17th International Conference on e-Health Networking, Applications and [40] S. Mattern. Mission Control: A history of the urban dashboard. Places. The
Services, pp. 439–442. IEEE, Aug. 2015. Journal of Public Scholarship on Architecture, Landscape and Urbanism,
[16] A. Faiola, P. Srinivas, and B. Doebbeling. A Ubiquitous Situation-Aware pp. 1–20, Mar. 2015.
Data Visualization Dashboard to Reduce ICU Clinician Cognitive Load. [41] G. McArdle and R. Kitchin. The Dublin Dashboard: Design and devel-
In th International Conference on e-Health Networking, Applications and opment of a real-time analytical urban dashboard. ISPRS Annals of the
Services, pp. 439–442. IEEE, Aug. 2015. Photogrammetry, Remote Sensing and Spatial Information Sciences, 4:19,
[17] A. Faiola, P. Srinivas, and J. Duke. Supporting Clinical Cognition: A 2016.
Human-Centered Approach to a Novel ICU Information Visualization [42] C. McCoy and H. Rosenbaum. Unintended and shadow practices of deci-
Dashboard. In AMIA Annual Symposium, pp. 560–569. American Medical sion support system dashboards in higher education institutions. vol. 54,
Informatics Association, Sept. 2015. pp. 757–759. Washington, DC, Oct. 2017.
[18] S. Few. Information Dashboard Design: The Effective Visual Communica- [43] T. H. S. Morgan, D. Murphy, and B. Horwitz. Police Reform Through
tion of Data. O’Reilly, Cambridge, MA, 2006. Data-Driven Management. In Police Quarterly, pp. 275–294, May 2017.
[19] S. Few. Blog post: There’s nothing mere about semantics. https://www. [44] A. Mottus, S. Graf, N.-S. Chen, et al. Use of dashboards and visualization
perceptualedge.com/blog/?p=2793, 2017. techniques to support teacher decision making. In Ubiquitous Learning
[20] S. Few and P. Edge. Dashboard confusion revisited. https: Environments and Technologies, pp. 181–199. Springer, 2015.
//www.perceptualedge.com/articles/visual_business_ [45] K. O’Connell, Y. Lee, F. Peer, S. Staudaher, A. Godwin, M. Madden, and
intelligence/dboard_confusion_revisited.pdf, 2007. E. Zegura. Making public safety data accessible in the westside Atlanta
[21] D. Filonik, R. Medland, M. Foth, and M. Rittenbruch. A Customisable data dashboard. In Bloomberg Data for Good, pp. 1–7. New York, Sept.
Dashboard Display for Environmental Performance Visualisations. In 2016.
Persuasive Technology, vol. 7822, pp. 51–62. Springer Berlin Heidelberg, [46] Z. Qu and J. Hullman. Keeping multiple views consistent: Constraints,
2013. doi: 10.1007/978-3-642-37157-8 8 validations, and exceptions in visualization authoring. IEEE transactions
[22] M. Galesic and R. Garcia-Retamero. Graph literacy: A cross-cultural on visualization and computer graphics, 24(1):468–477, 2018.
comparison. Medical Decision Making, 31(3):444–457, 2011. [47] L. Raftree. 13 tips on data dashboards for decision-making.
 https://lindaraftree.com/2015/06/05/13-tips-on-data-dashboards-for-
decision-making/, 2018.
[48] K. Rall, M. L. Satterthwaite, A. V. Pandey, J. Emerson, J. Boy, O. Nov,
and E. Bertini. Data Visualization for Human Rights Advocacy. Journal
of Human Rights Practice, 8(2):171–197, July 2016.
[49] R. M. Ratwani and A. Fong. connecting the dots: leveraging visual
analytics to make sense of patient safety event reports. Journal of the
American Medical Informatics Association, 22(2):312–317, 2014.
[50] M. Riphagen. Supporting reflective learning for daily activities using an
interactive dashboard. In ICERI2013 Proceedings, pp. 3239–3249. IATED,
2013.
[51] L. D. Roberts, J. A. Howell, and K. Seaman. Give Me a Customizable
Dashboard: Personalized Learning Analytics Dashboards in Higher Ed-
ucation. Technology, Knowledge and Learning, 22(3):317–333, June
2017.
[52] T. Rogers and A. Feller. Discouraged by peer excellence: Exposure
to exemplary peer performance causes quitting. Psychological Science,
27(3):365–374, Mar. 2016.
[53] H. J. Schulz, T. Nocke, M. Heitzler, and H. Schumann. A design space
of visualization tasks. IEEE Transactions on Visualization and Computer
Graphics, 19(12):2366–2375, 2013. doi: 10.1109/TVCG.2013.120
[54] H. Tangmunarunkit, C.-K. Hsieh, B. Longstaff, S. Nolen, J. Jenkins,
C. Ketcham, J. Selsky, F. Alquaddoomi, D. George, J. Kang, et al. Ohmage:
A general and extensible end-to-end participatory sensing platform. ACM
Transactions on Intelligent Systems and Technology (TIST), 6(3):38, 2015.
[55] J. Tucker. How is your school doing? online dashboard gives parents new
tool. https://www.sfgate.com/education/article/How-is-your-school-doing-
California-School-11001927.php.
[56] O. Velcu-Laitinen and O. M. Yigitbasioglu. The Use of Dashboards in Per-
formance Management: Evidence from Sales Managers. The International
Journal of Digital Accounting Research, 12:36–58, 2012.
[57] K. Verbert, E. Duval, J. Klerkx, S. Govaerts, and J. L. Santos. Learning
Analytics Dashboard Applications. In American Behavioral Scientist, pp.
1500–1509, Sept. 2013.
[58] K. Verbert, S. Govaerts, E. Duval, J. L. Santos, F. V. Assche, G. Parra,
and J. Klerkx. Learning dashboards: an overview and future research
opportunities. Personal and Ubiquitous Computing, 18(6):1499–1514,
Aug. 2014. doi: 10.1007/s00779-013-0751-2
[59] S. Wakeling, P. Clough, J. Wyper, and A. Balmain. Graph literacy and
business intelligence: Investigating user understanding of dashboard data
visualizations. Business Intelligence Journal, 20(4):8–19, 2015.
[60] J. Weiner, V. Balijepally, and M. Tanniru. Integrating Strategic and Op-
erational Decision Making Using Data-Driven Dashboards: The Case of
St. Joseph Mercy Oakland Hospital. Journal of Healthcare Management,
60(5):319.
[61] S. Wexler, J. Shaffer, and A. Cotgreave. The Big Book of Dashboards:
Visualizing Your Data Using Real-World Business Scenarios. John Wiley
& Sons, 2017.
[62] M. Whooley, B. Ploderer, and K. Gray. On the Integration of Self-tracking
Data amongst Quantified Self Members. pp. 151–160, Sept. 2014. doi: 10.
14236/ewic/hci2014.16
[63] B. A. Wilbanks and P. A. Langford. A Review of Dashboards for Data
Analytics in Nursing. In CIN: Computers, Informatics, Nursing, pp. 545–
549, Nov. 2014.
[64] B. Williamson. Digital education governance: data visualization, predic-
tive analytics, and ‘real-time’ policy instruments. In Journal of Education
Policy, pp. 1–19. Routledge, Jan. 2016.
[65] O. M. Yigitbasioglu and O. Velcu. A review of dashboards in performance
management: Implications for design and research. International Journal
of Accounting Information Systems, 13(1):41–59, Mar. 2012.
[66] J. Zeng and K. W. Glaister. Value creation from big data: Looking inside
the black box. Strategic Organization, 36(4):147612701769751–36, Nov.
2017.