Intro Vis/Gfx Interaction Evaluation Wrap-up

Thinking Interactively with Visualizations

Remco Chang

UNC Charlotte
Charlotte Visualization Center
 Intro Vis/Gfx Interaction Evaluation Wrap-up

Definition of Visual Analytics

• Visual analytics is the science of analytical reasoning

facilitated by interactive visual interfaces [Thomas &
Cook 2005]

• Since 2004, the field has grown

significantly. Aside from tens to
hundreds of domestic and
international partners, it now
has a IEEE conference (IEEE
VAST), an NSF program
(FODAVA), and a forthcoming
IEEE Transactions journal.
 Intro Vis/Gfx Interaction Evaluation Wrap-up

Individually Not Unique

• Interaction Design
• Cognitive Psychology
• Intelligence Analysis
• etc.
Analytical
Reasoning
• Data Mining and
• Machine Interaction
Learning Data • InfoVis
• Databases Visual • SciVis
Representation
• Information Representation • Graphics
Transformation
Retrieval • etc
• etc

Production,
Validation and
Presentation
Evaluation
Dissemination

• Tech Transfer • Quality Assurance

• Report Generation • User studies (HCI)
• etc • etc
 Intro Vis/Gfx Interaction Evaluation Wrap-up

In Combinations of 2 or 3…

Analytical
Reasoning
• Data Mining and
• Machine Interaction
Learning Data • InfoVis
• Databases Visual • SciVis
Representation
• Information Representation • Graphics
Transformation
Retrieval • etc
• etc

Production,
Validation and
Presentation
Evaluation
Dissemination
 Intro Vis/Gfx Interaction Evaluation Wrap-up

In Combinations of 2 or 3…
• Interaction Design
• Cognitive Psychology
• Intelligence Analysis
• etc.
Analytical
Reasoning
and
Interaction
Data
Visual
Representation
Representation
Transformation

Production,
Validation and
Presentation
Evaluation
Dissemination

• Tech Transfer
• Report Generation
• etc
 Intro Vis/Gfx Interaction Evaluation Wrap-up

This Talk Focuses On…

• Interaction Design
• Cognitive Psychology
• Intelligence Analysis
• etc.
Analytical
Reasoning
and
Interaction
Data • InfoVis
Visual • SciVis
Representation
Representation • Graphics
Transformation
• etc

Production,
Validation and
Presentation
Evaluation
Dissemination

• Quality Assurance
• User studies (HCI)
• etc
 Intro Vis/Gfx Interaction Evaluation Wrap-up

Interactive Analysis + Visualization

• Most people in the visualization community believe that
interactivity is essential for visualization and visual analytics:
– “A [visual] analysis session is more of a dialog between the analyst
and the data… the manifestation of this dialog is the analyst’s
interactions with the data representation” [Thomas & Cook 2005]
– “Without interaction, [a visualization] technique or system becomes
a static image or autonomously animated images” [Yi et al. 2007]
• The goal of this talk is to consider the role of interaction in
computer graphics, information visualization, and visual
analytics.
• First, we consider a stereotypical graphics application and try
adding interaction to it..
 Intro Vis/Gfx Interaction Evaluation Wrap-up

Urban Simplification

• (left) Original model, 285k polygons

• (center) e=100, 129k polygons (45% of original)
• (right) e=1000, 53k polygons (18% of original)

R. Chang et al., Legible simplification of textured urban models. IEEE Computer Graphics and Applications, 28(3):27–36, 2008.
R. Chang et al., Hierarchical simplification of city models to maintain urban legibility. ACM SIGGRAPH 2006 Sketches, page 130 , 2006.
 Intro Vis/Gfx Interaction Evaluation Wrap-up

Urban Simplification
• Which polygons to remove?

Original Model Simplified Model

using QSlim

Our Textured Model Our Model

Visually different, but quantitatively similar!

 Intro Vis/Gfx Interaction Evaluation Wrap-up

Urban Simplification

• The goal is to retain the “Image of the City”

• Based on Kevin Lynch’s concept of “Urban Legibility”

[1960]

– Paths: highways, railroads

– Edges: shorelines, boundaries
– Districts: industrial, historic
– Nodes: Time Square in NYC
– Landmarks: Empire State building
 Intro Vis/Gfx Interaction Evaluation Wrap-up

Urban Visualization with Semantics

• How do people think about a city?
– Describe New York…
• Response 1: “New York is large, compact, and crowded.”
• Response 2: “The area where I live there has a strong mix of
ethnicities.”

Geometric, Information, View Dependent (Cognitive)

 Intro Vis/Gfx Interaction Evaluation Wrap-up

Urban Visualization
• Geometric
– Create a hierarchy of shapes based on the rules of legibility
• Information
– Matrix view and Parallel Coordinates show relationships between clusters and
dimensions
• View Dependence (Cognitive)
– Uses interaction to alter the position of focus

R. Chang et al., Legible cities: Focus-dependent multi-resolution visualization of urban relationships. IEEE Transactions on Visualization
and Graphics , 13(6):1169–1175, 2007
 Intro Vis/Gfx Interaction Evaluation Wrap-up

The Role of Interaction in Visualization

• We can use interactions to… [Yi et al. 2007]
– Select: mark something as interesting
– Explore: show me something else
– Reconfigure: show me a different arrangement
– Encode: show me a different representation
– Abstract/Elaborate: show me more or less detail
– Filter: show me something conditionally
– Connect: show me related items

• In other words, we can use interactions to think.

 Intro Vis/Gfx Interaction Evaluation Wrap-up

(1) WireVis: Financial Fraud Analysis

• In collaboration with Bank of America

– Looks for suspicious wire transactions
– Currently beta-deployed at WireWatch
– Visualizes 15 million transactions over 1 year

• Uses interaction to coordinate four perspectives:

– Keywords to Accounts
– Keywords to Keywords
– Keywords/Accounts over Time
– Account similarities (search by example)
 Intro Vis/Gfx Interaction Evaluation Wrap-up

(1) WireVis: Financial Fraud Analysis

Search by Example
Heatmap View (Find Similar
(Accounts to Keywords Accounts)
Relationship)

Keyword Network
(Keyword
Relationships)

Strings and Beads

(Relationships over Time)

R. Chang et al., Scalable and interactive visual analysis of financial wire transactions for fraud detection. Information
Visualization,2008.
 Intro Vis/Gfx Interaction Evaluation Wrap-up

(2) Investigative GTD

• Collaboration with U. Maryland’s DHS Center of Excellence

START (Study of Terrorism And Response to Terrorism)
– Global Terrorism Database (GTD)
– International terrorism activities from 1970-1997
– 60,000 incidents recorded over 120 dimensions

• Visualization is designed to be “investigative” in that it is

modeled after the 5 W’s:
– Who, what, where, when, and [why]
– Interaction allows the user to adjust one or more of the W’s and
see how that affects the other W’s
 Intro Vis/Gfx Interaction Evaluation Wrap-up

(2) Investigative GTD

Who
Where

What

Evidence
Box

Original
Data When

R. Chang et al., Investigative Visual Analysis of Global Terrorism, Journal of Computer Graphics Forum (Eurovis), 2008.
 Intro Vis/Gfx Interaction Evaluation Wrap-up

(2) Investigative GTD:

Revealing Global Strategy
This group’s attacks
are not bounded by
geo-locations but
instead, religious
beliefs.
Y
WH
? Its attack patterns
changed with its
developments.
 Intro Vis/Gfx Interaction Evaluation Wrap-up

(2) Investigative GTD:

Discovering Unexpected Temporal Pattern
A geographically-
bounded entity in the
Philippines.
The ThemeRiver shows Domestic Group
its rise and fall as an
entity and its modus
operandi.
 Intro Vis/Gfx Interaction Evaluation Wrap-up

(3) iPCA: Interactive PCA

• Quick Refresher of Principle Component Analysis (PCA)
– Find most dominant eigenvectors as principle components
– Data points are re-projected into the new coordinate system
• For reducing dimensionality
• For finding clusters

• For many (especially novices), PCA is easy to understand

mathematically, but difficult to understand “semantically”.
 Intro Vis/Gfx Interaction Evaluation Wrap-up

(3) iPCA: Interactive PCA

R. Chang et al., iPCA: An Interactive System for PCA-based Visual Analytics. Computer Graphics Forum (Eurovis), 2009. To Appear.
 Intro Vis/Gfx Interaction Evaluation Wrap-up

(3) Evaluation – iPCA vs. SAS/INSIGHT

• Results
– A bit more accurate
– People don’t “give up”
– Not faster

• Overall preference
– Using letter grades (A
through F) with “A”
representing excellent and
F a failing grade.
 Intro Vis/Gfx Interaction Evaluation Wrap-up

If (Interactions == Thinking)…

• What is in a user’s interactions?

• If (interactions == thinking), what can we learn

from the user’s interactions?

• Is it possible to extract “thinking” from

“interactions”?
 Intro Vis/Gfx Interaction Evaluation Wrap-up

What is in a User’s Interactions?

Keyboard, Mouse, etc

Input
Visualization Human
Output

Images (monitor)

• Types of Human-Visualization Interactions

– Word editing (input heavy, little output)
– Browsing, watching a movie (output heavy, little input)
– Visual Analysis (closer to 50-50)
 Intro Vis/Gfx Interaction Evaluation Wrap-up

What is in a User’s Interactions?

• Goal: determine if there really is “thinking” in a user’s
interactions.

Grad
Compare! Students
Analysts (manually) (Coders)

Strategies Guesses of
Methods Analysts’
Findings thinking

Logged
(semantic)
Interactions
WireVis Interaction-Log Vis
 Intro Vis/Gfx Interaction Evaluation Wrap-up

What’s in a User’s Interactions

• From this experiment, we find that interactions contains at least:

– 60% of the (high level) strategies
– 60% of the (mid level) methods
– 79% of the (low level) findings

R. Chang et al., Recovering Reasoning Process From User Interactions. IEEE Computer Graphics and Applications, 2009.
R. Chang et al., Evaluating the Relationship Between User Interaction and Financial Visual Analysis. IEEE Symposium on VAST, 2009.
 Intro Vis/Gfx Interaction Evaluation Wrap-up

What’s in a User’s Interactions

• Why are these so much

lower than others?
– (recovering “methods” at
about 15%)

• Only capturing a user’s

interaction in this case is
insufficient.
 Intro Vis/Gfx Interaction Evaluation Wrap-up

Lessons Learned
• We have proven that a great deal of an analyst’s “thinking” in using a
visualization is capturable and extractable.

• Using semantic interaction capturing, we might be able to collect all

the thinking of expert analysts and create a knowledge database that
is useful for
– Training: many domain specific analytics tasks are difficult to teach
– Guidance: use existing knowledge to guide future analyses
– Verification, and validation: go back and check to see if everything was done
right.

• But not all visualizations are interactive, and not all thinking is
reflected in the interactions.
– A model of how and what to capture in a visualization for extracting an
analyst’s thinking process is necessary.
 Intro Vis/Gfx Interaction Evaluation Wrap-up

Conclusion

• Interactions are important for visualization and visual

analysis
– In considering interactions, one must be aware of the
necessary speed and frame rate of the displays.
• Techniques such as simplification, LOD, or approximation can be used.
– Interactions have been proven to help the understanding of
complex problems.
• Relevant interactions have been integrated in multiple visualizations
for different domains and demonstrated significant impact.
– Capturing and storing analysts’ interactions have great
potential
• They can be aggregated to become a “knowledge database” that has
traditionally been difficult to create manually.
 Intro Vis/Gfx Interaction Evaluation Wrap-up

Discussion

• What interactivity is not good for:

– Presentation
– YMMV = “your mileage may vary”
• Reproducibility: Users behave differently each time.
• Evaluation is difficult due to opportunistic discoveries..
– Often sacrifices accuracy
• iPCA – SVD takes time on large datasets, use iterative
approximation algorithms such as onlineSVD.
• WireVis – Clustering of large datasets is slow. Either
pre-compute or use more trivial “binning” methods.
 Intro Vis/Gfx Interaction Evaluation Wrap-up

Discussion
• Interestingly,
– It doesn’t save you time…
– And it doesn’t make a user more
accurate in performing a task.
• However, there are empirical
evidence that using interactivity:
– Users are more engaged (don’t give
up)
– Users prefer these systems over
static (query-based) systems
– Users have a faster learning curve
• We need better measurements to
determine the “benefits of
interactivity”
 Intro Vis/Gfx Interaction Evaluation Wrap-up

Future Work
• Interaction Design
• Cognitive Psychology
• Intelligence Analysis
• etc.
Analytical
Reasoning
• Data Mining and
• Machine Interaction
Learning Data • InfoVis
• Databases Visual • SciVis
Representation
• Information Representation • Graphics
Transformation
Retrieval • etc
• etc

Production,
Validation and
Presentation
Evaluation
Dissemination

• Tech Transfer • Quality Assurance

• Report Generation • User studies (HCI)
• etc • etc
 Intro Vis/Gfx Interaction Evaluation Wrap-up

Future Work

• Lots of possible combinations. Are they all

meaningful?
• Of particular interest to me is “Data +
Interaction + Visualization”
– How to apply computational approaches to find
solutions that are usable by humans?
• Linear (PCA) and non-linear (manifold learning) create
dimensions that are semantically difficult to define
• Nodes within a Bayesian network are difficult to
comprehend, therefore the results difficult to take at
face value.
Intro Vis/Gfx Interaction Evaluation Wrap-up

Thank you!

[email protected]
http://www.viscenter.uncc.edu/~rchang