Classifying Desirable Features of Software Visualization Tools for Corrective

Maintenance
Mariam Sensalire∗ Patrick Ogao† Alexandru Telea‡
Faculty of Computing and IT Faculty of Computing and IT Institute of Mathematics and Computing Science
Makerere University, Uganda Makerere University, Uganda University of Groningen, the Netherlands

Abstract and tools is quite broad. Since a tool’s audience strongly depends
on its purposes [Maletic et al. 2002], evaluating similar-purpose
We provide an evaluation of 15 software visualization tools appli- tools would be more insightful [Koschke 2003]. Here, Storey et al.
cable to corrective maintenance. The tasks supported as well as [Storey and German 2005] compared 12 tools that provide aware-
the techniques used are presented and graded based on the support ness of human activities during software development against the
level. By analyzing user acceptation of current tools, we aim to help categories of intent, information, presentation, interaction and ef-
developers to select what to consider, avoid or improve in their next fectiveness. In corrective maintenance, Baecker et al. analyzed
releases. Tool users can also recognize what to broadly expect (and three classes of SoftVis techniques used for debugging [Baecker
what not) from such tools, thereby supporting an informed choice et al. 1997]: animation, improved typographic representations, and
for the tools evaluated here and for similar tools. error sonification. Earlier, we evaluated ten general-purpose Soft-
Vis software-understanding tools[Sensalire and Ogao 2007a]. This
evaluation forms the basis of our extended study of SoftVis tools
1 Introduction for CM.
Several studies have advocated the use of software visualization
tools for corrective maintenance (CM) [Baecker et al. 1997; Swan- 3 Classification Model
son 1976]. However, although several tool surveys exist on the In-
ternet, it is still hard to answer the questions ”does this tool fit my We classify SoftVis tools using four categories of desirable fea-
user profile, context, and needs?” (for tool users) and ”what do the tures: Effectiveness, Tasks supported, Techniques used and Avail-
users most (dis)like in a tool?” (for tool developers). In a previ- ability. These features and the scales their presence is measured
ous study [Sensalire and Ogao 2007b], we collected feedback from on are derived from several user interviews [Sensalire and Ogao
software engineers who used such tools for program understand- 2007b]. We use either a low,medium,high scale or a simpler yes,no
ing in general, and extracted several desirable features of SoftVis scale, as described next.
tools. In this paper, we refine and focus the set of desirable features
on SoftVis tools for CM. We aim to guide users in selecting Soft- 3.1 Category A: Effectiveness
Vis tools for CM, based on their desirable features, either from the
tools discussed here or from a larger, more general, set. Next, we
aim to discover possible correlations between perceived tool accep- An effective tool arguably helps users to solve the problems it was
tance levels and the usage of certain visual techniques, to further designed to assist with. Effectiveness is task-specific, hence hard to
clarify what makes a tool accepted (or not). measure in general. Yet, we identified three non-functional proper-
ties that effective SoftVis tools should have, as follows.
This paper is organized as follows. Section 2 overviews related
work. Section 3 presents our classification model for desirable tool Scalability: A scalable tool supports CM tasks on systems of mil-
features. Section 4 presents the evaluated tools and evaluation pro- lions of LOC and/or thousands of classes and/or source files. Tools
cedure (Sec. 5). Section 6 discusses our evaluation results. Sec- created for educational purposes, proof-of-concept, or research pro-
tion 7 concludes the paper. totypes are given low; high if they support large-scale code (as de-
fined above); and medium if falling in between.
2 Related Work Integration: This measures how easily several tools can switched
and exchange data to complete a given task in an IDE or similar
Price et al. [Price et al. 1993] compared 12 tools against 6 desir- setup. A tool that input and output data from/to other tools, and
able features categories: scope, content, form, method, interaction also senses and displays data changes, is given high. Tools that co-
and effectiveness. The tools were however not related to a single exist in an IDE but do not actively exchange data and/or events with
application area. Maletic et al. [Maletic et al. 2002] compared 5 peer tools are given medium. Standalone tools are given low.
software tools along 5 axes: task, audience, target, representation,
and medium. Similar to Price et al., the scope of this taxonomy Query support: Tools that work in batch mode are given low. Tools
that support just lexical queries get medium. Tools that corre-
∗ e-mail: [email protected] late queries and visualizations (query highlighting) or use complex
† e-mail: [email protected] queries(e.g., syntax-aware or type-based) get high.
‡ [email protected]

3.2 Category B: Tasks supported

We consider the following tasks: detecting code smells, code refac-

toring, trace analysis, and debugging support. While not exhaus-
tive, these are typical for CM, and also well understood by our in-
terviewed users.
Detecting code smells / Refactoring: Code smells signal bad pro-
gramming, design, or code, and are often used to drive refactoring.
Tools that support neither refactoring nor code smells detection, e.g.
debuggers, get low. Tools that can detect smells but do not suggest
how to correct these, e.g. static analyzers, get medium. Tools that CodeRush with RefactorPro is a Visual Studio add-on that enables
detect and suggest how to correct smells get high. code refactoring by providing change options with hints visually
within the code.
Trace analysis: Traces are defined as data gathered during a pro-
gram’s execution. Tools that can neither generate nor show traces CodePro AnalytiX is an Eclipse plugin that provides code coverage
get low. Tools that generate but cannot display traces, e.g. profilers, analysis, dependency analysis, and visual report generation.
or display but not generate traces, e.g. data viewers, get medium.
Tools that generate and display traces get high. Code Coverage is a Java NetBeans plugin that colors code based on
low coverage.
Debugging support: Tools that do basic debugging, e.g. break-
points, code stepping, and simple text watches, get low. Tools that CVSgrab uses customizable dense pixel displays to correlate project
show higher-level debugging facts, e.g. bug or testcase data, get activity from code repositories and bug data from a Bugzilla
medium. Tools that show both low and high-level debugging data, database.
or relate bugs to actions carried out to remove them, get high. Gammatella does remote monitoring by combining execution-
metric-colored treemaps and SeeSoft-like views.
3.3 Category C: Availability
JBIXBE simplifies debugging of multi-threaded code, combining
Availability relates to the programming languages a tool supports, classical breakpoints, step-mode execution, and watches with exe-
whether it is free or commercial, and the platform on which it runs. cution flowcharts and UML-like class diagrams.
A tool must be available to evaluate it, so we do not grade this JIVE supports runtime analysis and code debugging using forward
aspect, but just provide a textual description. and backward stepping and visualizes execution history using inter-
active UML sequence diagrams.
3.4 Category D: Techniques used
JSwat provides debugging by a combination of standard IDE-like
Assessing the types (and presence degree) of techniques used in a breakpoints, watches, and visualizations.
SoftVis tool can shed light on usability and ease of learning. Fol- Paraver visualizes program traces using 2D dense pixel displays
lowing previous taxonomies [Maletic et al. 2002] and user feed- similar to CVSgrab.
back [Sensalire and Ogao 2007b], we consider the following tech-
niques: Project Analyzer is an IDE that uses syntax highlighting and multi-
ple code views to aid checking for error proneness as well as guid-
2D or 3D Graphics 2D graphics (present: yes-no) is sometimes ing on ways in which to improve the code.
seen as less expressive than 3D, but can be easier to learn and use.
3D graphics, measured on the same scale, can display more data, Source Navigator provides syntax-highlighed and graph-based
but may come with additional usability costs. views to show code symbols and syntax relations, and also several
grep-based queries.
Animation: Animation can be used to show dynamic data, e.g.
traces or code evolution. Effects such as smooth zooming and pan- STAN is an Eclipse plugin offering linked graph views, metric-
ning are here not considered animations. colored treemaps, and metric histograms to understand and detect
design flaws.
Color usage: Tools that do not use color, e.g. command-line ones,
are given low. Using a few saturated hues, e.g., to display categori- Tarantula helps finding faults or problems at code level, using a
cal values, gives a medium. Tools that use blending for multivariate Seesoft-like code coloring to show test passes or fails.
data display or color mapping of continuous value ranges get high.
VB Watch provides code testing and debugging using multiple text
User interaction: Tools that show static views get low. Tools that views, metrics histograms, and metric-annotated call graphs.
support a single task via user interaction, e.g., adjust a layout with
the mouse [Source-Navigator-Team 2007], get medium. Tools that
support two or more tasks by two or more medium-graded tech- 5 Tools Evaluation
niques, e.g. direct mouse manipulation and rubberbanding, get
high. Our study participants were 16 professional developers, 12 male
and 4 female. Only two had used a SoftVis tool before (Table 1).
Multiple views: Tools that use a single view get low. Several not- We used a pre-study questionnaire to select users having good skills
linked views are gives a medium. Several linked views get a high. (over 4 years in maintenance) and being fluent in 2 or 3 program-
ming languages. We assigned tool groups Gi to users matching
Information density: Quantifies the amount of artifacts (e.g. code the tools’ programming language and platform to the ones best
lines, classes, functions, or files) drawn per screen unit. For exam- known by the respective user, as follows: VBWatch, CodeRush
ple, a text editor has low density, a UML browser medium density, (G1 ); JIVE, JBIXE, Source Navigator (G2 ); Alinea, Paraver, Code
and a dense-pixel treemap high density. Coverage (G3 ); Project Analyzer, CodePro Anaytix (G4 ); Taran-
Navigation: We measure navigation options (panning, scrolling, tula, Gammatella (G5 ); CVSgrab (G6 ); and JSwat, STAN (G7 ).
zooming and viewpoint change) using a yes,no scale. Four source code bases were studied: ArgoUML for Java tools,
Dynamic visualization: Dynamic visualizations can display data the network simulator ns 2 [Henderson 2008] for C++ tools, a pro-
generated on-the-fly while a program is running, e.g., debugging or prietary code base for VB tools, and the ArgoUML and Visual-
tracing data. We use here a yes,no scale. ization Toolkit repositories for CVSgrab. After a 15-minute tuto-
rial, the participants performed six generic tasks to assess the tools’
capabilities for (a)Scalability, (b)Integration, (c)Query support,
4 Evaluated Tools (d)Detecting code smells, (e)Trace analysis, and (f)Debugging, as
follows:
We evaluated our desirable features on the following tools.
a Analyze code for errors or error-proneness, from classes to
Allinea DDT debugs scalar, multi-threaded large-scale parallel ap- files to packages to the whole project;
plications. Visualization is used mainly to control program execu-
tion. b Change code using a tool, analyze effects using other tools;
 User Languages Years of Years in Tool group developer as the menus for the new tool are incorporated within the
known experience maintenance evauated
existing IDE. Standalone tools, on the other hand, present a new
1 VB, Java, C# 6 4 1
2 VB, C# 8 5 1
learning challenge to the user, thereby reducing the ease of use.
3 Java, C++ 8 5 2
The medium to high levels of integration observed in our evaluation
4 Java, C 9 7 2 certify the importance of this factor to the success of the tools. Tool
5 C, C++, Java, 7 5 3 developers would therefore benefit from integrating their tools in
6 C++, Java > 10 7 3 order to increase their acceptance by users.
7 C++, VB, Java, > 10 8 4
8 VB, Java, C# 7 5 4
Rich color usage, multiple views, rich user interaction and naviga-
9 C++, Java 8 5 5
tion were all features deemed extremely desirable, and also found
10 C, VB 6 4 5 to be well supported by most users.
11 C++, Java > 10 7 7
As programming language, Java is well supported by most tools,
12 C#, Java 9 6 7
probably due to the ease with which Java engines enable data ex-
13 C, Java, C++ 8 4 6
14 C, Java, C# 9 6 6
traction and analysis. Hence, future tool developers can choose to
15 C, Python, Java 8 4 6
support Java or may choose instead to exploit the lower support of
other languages, e.g. C++, so as to develop tools for them.
Table 1: Evaluation participants Visual scalability, or information density, is not high in most tools.
This can limit user acceptance. High information density allows
c Find all classes having errors and display them to find how one to monitor more at a given time, which is important for high-
they interact with the rest of the code; volume dynamic data such as traces, but also large code bases. Yet,
very high density tools, e.g. CVSgrab, were quite hard to learn.
d Find two code smells and use the tool’s refactoring hints to This may prove an initial acceptance blocker. Overall, we believe
improve them; tool developers should use multiscale methods to allow users to
flexibly select the data amount shown at a given time.
e Generate traces and analyze them;
f Set breakpoints, step through code and find who has been in- 6.1 Comparison with program comprehension studies
volved in debugging activities over the last 2 years;
We now correlate our results with [Vans et al. 1999] who observed
The users graded the perceived support level and visual techniques
developers during industrial CM tasks. The CM actions involved
offered (Sec. 3.4) and also wrote down any additional remarks.
chunking and hypothesis generation at different abstraction levels.
Hence, SoftVis tools that support CM should let users view more
6 Evaluation Results abstraction levels to enable hypothesis generation. To (dis)prove a
hypothesis, this further requires tool query support and query-view
Tables 2 and 3 show the desirable features and supported techniques integration. Inability to query large code also makes it hard to do
obtained by averaging the given grades for each tool. efficient chunking. Additional factors crucial in showing many ab-
straction levels are the multiple view support and IDE integration
Tool acceptance was heavily influenced by the amount of user inter- for inter-view navigation. Six of the 15 tools evaluated had high
action provided. Too little interaction impairs usability. Too much query support, while 8 tools had medium support. Most tools there-
interaction makes learning hard, thus affecting usability too. This fore tend towards high query support which is the prefered mode
correlates with our earlier study [Sensalire and Ogao 2007b], where for CM [Sensalire and Ogao 2007b; Vans et al. 1999]. Mainte-
users disliked tools asking a lot of input before showing any output. nance professionals had varying information needs depending on
Also, tools that did not enable users to tune at all the final images expertise [Vans et al. 1999], most experienced ones tending to favor
(layout, colors, annotations) were not desired. precise searches. This emphasizes the need for refined, high query
Over half of the participants questioned the need for SoftVis tools, support in SoftVis tools for CM. As program comprehension is an
and mentioned their preference for the Eclipse IDE, feeling that it essential part of (corrective) maintenance, a further comparison of
would suffice for their CM tasks. This suggests that (new) SoftVis SoftVis tools in this area with studies of program comprehension
tools should show clear advantages as compared to existing ’plain’ tools would bring additional information.
IDEs. An evaluation such as the ours is relevant in proving the level
to which a tool lives up to its claims. A lack of such studies may cre- 7 Conclusions
ate skepticism on the users’ side. Tool developers can use the users’
IDE attachment to their benefit, by creating tools that plug into an We evaluated fifteen SoftVis tools that support corrective mainte-
IDE, or using the IDE metaphor in organizing visualizations. For nance. Our aim was to detect patterns that can guide tool users and
example, the STAN tool was found natural as its visualizations fol- developers in their choice of tool to use or techniques to support.
low the Eclipse displays. Five of the participants felt that education The patterns observed from the evaluation were further correlated
had a major role in industrial tool adoption, saying that they would with results on a program comprehension study in CM. Several fea-
have been more motivated to use SoftVis tools in their work if they tures (IDE integration, scalability, multiple views, and query sup-
had been exposed to them during their undergraduate training. port) were strongly required by all users and provided by increas-
Among the considered tools, there is more support for static than ingly many tools. Some other features (3D, animation) were less
dynamicvisualization. Animation and 3D, on the other hand, are present. This raises several open questions:
not frequently utilized. Also, 2D visualizations were much better a Do users reject 3D and animation in SoftVis tools for CM or
accepted by nearly all users, as previously noticed are developers reluctant to provide them?
The preciseness of answering queries was ranked as very important. b Would developers exposed to SoftVis tools in their early train-
A tighter integration between analysis and presentation, as well as ing be more willing to use them in their professionalf work?
too integration, were deemed as extremely desirable by all users.
Tool integration has a direct relation to the tool’s user interface and c When developing SoftVis tools for CM, when should the
as such influences the tool’s usability. When a tool is integrated views of practicing professional be mainly involved: at the
with an IDE, a familiar user interface is provided to the software point of idea conception; during tool development; or when
 EFFECTIVENESS TASKS SUPPORTED AVAILABILITY
TOOLS Scalability Integration Query support Detect smells Trace analysis Debugging Languages Licensing Platform
Allinea DDT High Low High Medium Medium Medium C, C++, Fortran Comm. Linux
CodeRush Medium High Medium High Low Low VB, C# Comm. Win, Linux
CodePro Analytix High Medium High High Low Low Java Comm. Linux, Win
Code Coverage Medium High Medium Medium Low Medium Java Free Linux, Win
CVSgrab High High Medium Medium Low Medium Free Linux, Win
Gammatella Low Low High Medium High Medium Java Free Linux, Win
JBIXBE Medium Low Medium Low Low Medium Java Comm. Linux, Win
JIVE Medium Medium High Medium High Medium Java Free Linux, Win
JSwat Medium High Medium Medium Low Low Java Free Linux, Win
Paraver High Low High Medium High High Java Free Linux
Project Analyzer Medium Low Medium Medium Low Low VB Comm. Windows
Source Navigator High High Medium Medium Low Low Java, C/C++, Fortran Free Windows
STAN High Medium High Medium Medium Low Java Comm. Linux, Win
Tarantula Low Low Low Medium High Medium C Free Linux
VB Watch Medium Low Medium Medium Medium Medium VB 6 Comm. Win

Table 2: Tools against desirable features

VISUAL TECHNIQUES USED

TOOLS 2D 3D Animation Color usage User interaction Multi views Info. density Navigation Dynamic viz
Allinea DDT Yes Yes No Medium Low Medium Low Yes No
CodeRush Yes No Yes Medium Medium High Low Yes No
CodePro Analytix Yes No No Medium Medium High Medium Yes No
Code Coverage Yes No No Medium Medium Medium Low Yes No
CVSgrab Yes No No High High High High Yes No
Gammatella Yes No No High Medium High High Yes No
JBIXBE Yes No No Medium Medium High Medium Yes Yes
JIVE Yes No No Medium Medium High Medium Yes Yes
JSwat Yes No No Medium Medium High Low Yes No
Paraver Yes Yes No High High High High Yes Yes
Project Analyzer Yes No No Medium Medium Medium Medium Yes No
Source Navigator Yes No No Medium Medium High Medium Yes No
STAN Yes No No Medium Medium Medium Medium Yes No
Tarantula Yes No No High Low Medium High Yes Yes
VB Watch v2 Yes No No Medium Medium Medium Low Yes No

Table 3: Tools against techniques used

the tool is fully functional? Involvement at all points, al- S ENSALIRE , M., AND O GAO , P. 2007. Tool users require-
though desirable, may not be practically feasible. ments classification:how software visualization tools measure
up. AFRIGRAPH ’07’ Proceedings of the 5th International Con-
Future work includes refining the analysis on a set of specific tasks ference on Computer graphics, virtual reality, visualization and
and sample code bases in order to provide a more quantitative evalu- interaction in Africa, Grahamstown, South Africa.
ation of the tools’ abilities to address their tasks. A second direction
is to extend our evaluation of SoftVis tools to adaptive, perfective S ENSALIRE , M., AND O GAO , P. 2007. Visualizing object ori-
and preventive maintenance. ented software:towards a point of reference for developing tools
for industry. 4th IEEE International Workshop on Visualizing
Software for Understanding and Analysis, Banff, Canada.
References
S OURCE -NAVIGATOR -T EAM, 2007. Source code analysis tool.
BAECKER , R., D I G IANO , C., AND M ARCUS , A. 1997. Software Website. http://sourcenav.sourceforge.net/.
visualization for debugging. Commun. ACM 40, 4, 44–54. S TOREY, M.-A. D., AND G ERMAN , D. M. 2005. On the use of
H ENDERSON , T., 2008. ns-allinone-2.33 release. Website. visualization to support awareness of human activities in soft-
http://http://sourceforge.net/project/ ware development: a survey and a framework. In SoftVis ’05:
showfiles.php?group_id=149743&package_id= Proceedings of the 2005 ACM symposium on Software visualiza-
169689&release_id=588643. tion, ACM, New York, NY, USA, 193–202.

KOSCHKE , R. 2003. Software visualization in software mainte- S WANSON , E. B. 1976. The dimensions of maintenance. In ICSE
nance, reverse engineering, and re-engineering: a research sur- ’76’ Proceedings of the 2nd International Conference on Soft-
vey. Journal of Software Maintainance. Evol.: Res. Pract 15, ware Engineering, IEEE Computer Society Press, Los Alamitos,
87–109. CA, USA.
VANS , A., VON M AYRHAUSER , A., AND S OMLO , G. 1999.
M ALETIC , J., M ARCUS , A., AND C OLLARD , M. 2002. A task Program understanding behavior during corrective maintenance
oriented view of software visualization. Proceedings of IEEE of large-scale software. International Journal of Human-
Workshop of Visualizing Software for Understanding and Analy- Computers Studies 51, 1, 31–70.
sis Paris, France,, 32–40.
P RICE , A., BAECKER , R., AND S MALL , I. 1993. A principled tax-
onomy of software visualization. Journal of Visual Languages
and Computing 4(3):211-266.