-
Notifications
You must be signed in to change notification settings - Fork 1
/
Copy patheclemma20.html
301 lines (301 loc) · 17.4 KB
/
eclemma20.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta name="date" content="2017-03-28 22:12:41 +0200" />
<link rel="stylesheet" href="../book.css" type="text/css" />
<link rel="shortcut icon" href="../favicon.ico" type="image/x-icon" />
<title>EclEmma - EclEmma 2.0 Whitepaper</title>
</head>
<body>
<table cellspacing="0" cellpadding="0" border="0" style="width:100%;">
<tr>
<td style="padding:5px 10px 5px 10px;font-size:16pt;font-weight:bold;color:#000066;background-color:#f7f7f7;border:1px solid #a5b6c6;">
EclEmma <span style="color:#8998a5;">3.1.9</span>
</td>
<td style="padding:5px 10px 5px 10px;font-size:16pt;font-weight:bold;color:#8998a5;border-bottom:1px solid #a5b6c6;">
Java Code Coverage for Eclipse
</td>
<td style="padding:0px 0px 0px 0px;border-bottom:1px solid #a5b6c6;text-align:right">
<a href="https://marketplace.eclipse.org/marketplace-client-intro?mpc_install=264" title="Drag to your running Eclipse workspace to install EclEmma Java Code Coverage">
<img src="https://marketplace.eclipse.org/modules/custom/eclipsefdn/eclipsefdn_marketplace/images/btn-install.svg" alt="Drag to your running Eclipse workspace to install EclEmma Java Code Coverage" style="vertical-align:text-bottom; border:0px solid black" />
</a>
</td>
</tr>
<tr><td style="vertical-align:top;padding:20px 10px 20px 10px;background-color:#f7f7f7;border-left:1px solid #a5b6c6;border-right:1px solid #a5b6c6;border-bottom:1px solid #a5b6c6;">
<div class="menu" style="background-image:url(../images/topic.gif);margin-left:0px;">
<a href="../index.html">Overview</a>
</div>
<div class="menu" style="background-image:url(../images/container.gif);margin-left:0px;">
<a href="../installation.html">Installation</a>
</div>
<div class="menu" style="background-image:url(../images/container.gif);margin-left:0px;">
<a href="../userdoc/index.html">User Guide</a>
</div>
<div class="menu" style="background-image:url(../images/container.gif);margin-left:0px;">
<a href="../support.html">Support</a>
</div>
<div class="menu" style="background-image:url(../images/topic.gif);margin-left:0px;">
<a href="../resources.html">Resources</a>
</div>
<div class="menu" style="background-image:url(../images/container.gif);margin-left:0px;">
<a href="index.html">Developer Information</a>
</div>
<div class="menu" style="background-image:url(../images/topic.gif);margin-left:16px;">
<a href="architecture.html">Architecture</a>
</div>
<div class="menuhi" style="background-image:url(../images/topic.gif);margin-left:16px;">
<a href="eclemma20.html">EclEmma 2.0</a>
</div>
<div class="menu" style="background-image:url(../images/topic.gif);margin-left:16px;">
<a href="checklist.html">Release Checklist</a>
</div>
<div class="menu" style="background-image:url(../images/container.gif);margin-left:0px;">
<a href="../research/index.html">Research</a>
</div>
<div class="menu" style="background-image:url(../images/topic.gif);margin-left:0px;">
<a href="../jacoco/index.html">JaCoCo</a>
</div>
<div class="menu" style="background-image:url(../images/topic.gif);margin-left:0px;">
<a href="../changes.html">Change Log</a>
</div>
<div class="menu" style="background-image:url(../images/topic.gif);margin-left:0px;">
<a href="../license.html">License</a>
</div>
<div class="menu" style="background-image:url(../images/topic.gif);margin-left:0px;">
<a href="../contact.html">Contact</a>
</div>
<div class="menu" style="margin-top:15pt;background-image:url(../images/extern.gif)">
<a href="https://github.com/jacoco">GitHub Home</a>
</div>
<div style="text-align:center;margin-top:0pt;">
<a href="http://www.eclipse.org/org/press-release/20080318_AwardsWinners.php"><img src="../images/award.jpg" width="144" height="120" alt="Eclipse Community Award 2008" style="border:0px solid black" /></a>
</div>
<div style="text-align:center;margin-top:10pt;">
<a href="http://www.jacoco.org/jacoco"><img src="../images/jacoco.png" height="62" width="140" alt="JaCoCo" style="border:0px solid black" /></a>
</div>
<div style="text-align:center;margin-top:10pt;">
<a href="http://www.sonarqube.org/"><img src="../images/sonarqube.png" height="40" width="140" alt="Inspected with sonarqube" style="border:0px solid black" /></a>
</div>
<div style="text-align:center;margin-top:10pt;padding-top:15px;border-top:1px solid #a0a0a0">
<a href="https://www.mtrail.ch/en/career.html"><img src="../images/mtrail.png" width="140" height="200" alt="jobs at mtrail" style="border:0px solid black" /></a>
</div>
<div style="text-align:center;margin-top:10pt">
<a href="http://www.javaspecialists.eu/"><img src="../images/tjsn.jpg" width="140" height="140" alt="The Java Specialists' Newsletter" style="border:0px solid black" /></a>
</div>
</td><td style="vertical-align:top;padding:0px 0px 5px 20px;width:90%;border-bottom:1px solid #a5b6c6;" colspan="2">
<h1>EclEmma 2.0 Position Paper</h1><p>
<i><b>The <a href="../index.html">EclEmma</a> code coverage plug-in for
Eclipse originally was designed as a wrapper for the
<a class="extern" href="http://emma.sourceforge.net/">EMMA</a> code coverage
library to make EMMA available within the Eclipse IDE. Since its initial
release in 2006 EclEmma became a popular addition to the Eclipse Java IDE. As
EMMA maintenance has stopped some years ago and we wanted to see functional
enhancements like branch coverage for EclEmma we started a completely new
code coverage backend called
<a href="http://www.jacoco.org/jacoco">JaCoCo</a> under the
umbrella of the EclEmma project. The purpose of this paper is to setup
expectations for the EclEmma/JaCoCo user community and serve as a guide line
for technical decisions. The first release fully backed by JaCoCo is
"EclEmma 2.0".</b></i>
</p><p style="text-align:right;margin-bottom:15pt">
Marc R. Hoffmann, Mountainminds GmbH & Co. KG, September 2011 (updated)
</p><h2>Starting Point and Maxims</h2><p>
Since its initial release in 2006 EclEmma became one of the most widely uses
code coverage tool in the Java developer's tool box and is one of the
<a class="extern" href="http://marketplace.eclipse.org/metrics/installs">most installed</a>
Eclipse plug-ins. As EMMA development has stopped shortly after the first
EclEmma release in 2006 we started working on an alternative called JaCoCo in
2009. The aim of the <a href="../jacoco/index.html">JaCoCo Project</a> is
<i>providing a lightweight, flexible and well documented library for
integration with various build and development tools</i>, see this
<a href="http://www.jacoco.org/jacoco/trunk/doc/mission.html">mission statement</a>
for details. In the meantime JaCoCo is fully functional and has been adopted
by several projects. For example the Eclipse platform integration builds are
analyzed using the JaCoCo Ant tasks. Many open source as well as commercial
projects use the
<a class="external" href="http://docs.codehaus.org/display/SONAR/JaCoCo+Plugin">JaCoCo plug-in for Sonar</a>
to control test quality.
</p><p>
As EclEmma was originally developed to provide code coverage analysis directly
for the individual developer, the same now applies for JaCoCo. A consistent
view to test coverage metrics in the automated builds as well as the local
Eclipse IDE requires a JaCoCo plug-in for Eclipse. From the EclEmma project's
point of view there are at least the following three paths towards a JaCoCo
integration for Eclipse:
</p><ul>
<li><b>Option 1:</b> Implement a new and separate Eclipse plug-in for JaCoCo.</li>
<li><b>Option 2:</b> Create a plugable architecture that allows to integrate
different code coverage technologies.</li>
<li><b>Option 3:</b> Migrate EclEmma towards JaCoCo (preferred).</li>
</ul><p>
I propose to follow option 3, which means migrate the existing EclEmma project
towards JaCoCo. Given the high exposure of EclEmma this options offers a set
of benefits:
</p><ul>
<li>Smooth upgrade path to the new version of EclEmma.</li>
<li>Existing functionality and features of EclEmma like "one-click" launches
and direct visual feedback within the IDE are kept.</li>
<li>Proven and robust code base.</li>
</ul><p>
The other two options appear less attractive due to two main aspects: The
first point is the EMMA library itself. Given the issues described before and
the fact that EMMA fails with Java 7 byte code the future use of this great
but abandoned library is questionable. Therefore maintaining a EMMA-only
plug-in does not look reasonable in the long run. Users explicitly bound to
EMMA may continue using the EclEmma 1.x stream (see below).
</p><p>
Providing a plugable architecture for different coverage libraries looks like
best practice at the first glance. This is how the overall Eclipse platform
has been successfully crafted. When looking at different code coverage
libraries features and integration strategies are very different. For
example EMMA requires a huge overhead for instrumentation of local class files
and hooks to tweak class paths. JaCoCo in contrast does not require this but
gives additional coverage metrics and source highlighting for branch coverage.
A <i>code coverage framework</i> would therefore either be
</p><ul>
<li>very small as the common overlapping is small, or</li>
<li>quite big containing several base services (like class path re-writing)
for particular code coverage libraries.</li>
</ul><p>
On the other hand the Eclipse launch and debug framework already is a
plugable architecture that allows different integrations for different code
coverage technologies even today.
</p><h2>Functional Improvements</h2><p>
For EclEmma 2.0 the following functional improvements are planned:
</p><h3>Branch Coverage</h3><p>
This additional metric will show coverage of all decision points in the
program flow due to <code>if</code> and <code>switch</code> statements or the
<code>?</code> operator. The result will be displayed directly in the Java
editors similar to the JaCoCo reports using a new annotation icon in the left
ruler. In addition the <i>Coverage</i> view can be switched to branch
counters. Also the <i>Coverage</i> property page will show branch figures for
all Java elements. In addition cyclomatic complexity is shown in the view as
well as on the property page. The former "block" counters will not be
supported any more.
</p><h3>Faster Launching</h3><p>
Due the way how the JaCoCo coverage library works there will be no additional
delay any more when applications under test are launched. This is an
significant performance improvement especially for large applications and test
suites.
</p><h3>Less Invasive</h3><p>
Certain launch types and test scenarios require so called <i>in-place
instrumentation</i> in EclEmma 1.x. With this option the original class files
get modified on disk and need to be restored with a clean build when switching
back to another launch mode. Also JAR files where excluded from coverage
analysis when <i>in-place</i> mode is activated. Without <i>in-place
instrumentation</i> the class path of the application under test was modified
which caused trouble for some applications.
</p><p>
With EclElmma 2.0 class files on disk will never be modified and the class
path of an applications stays untouched. This will remove several hassles
especially for Eclipse application launches and JUnit plug-in tests.
</p><h3>Intermediate and Remote Coverage Analysis</h3><p>
JaCoCo 2.0 (or more likely one of its subsequent versions) will support
intermediate coverage dumps on applications running locally or remote without
stopping the applications under test. This will also allow to reset the
collected coverage information for a running application.
</p><h3>Flexible Analysis Scope</h3><p>
The scope of a coverage analysis can be modified at any time afterwards. While
in EclEmma 1.x the scope needs to be specified on the coverage launch dialog
<i>before</i> the application is launched, with EclEmma 2.0 the scope can be
altered at any time when the result of the coverage session is viewed.
</p><h2>The Plug-in Project Name</h2><p>
Back in 2006 <i>EclEmma</i> originally was an acronym formed from the project
names Eclipse and EMMA. Today the name is widely used for code coverage in
Eclipse without an focus on the technology used internally. Therefore the name
should be kept, also for the plug-in Ids and Java package names. Which
directly leads to the next point.
</p><h2>Update from EclEmma 1.x to EclEmma 2.0</h2><p>
Once EclEmma 2.0 has been stabilized and the first release is published
EclEmma users should be able to get the new JaCoCo based implementation simply
through the regular update site process.
</p><h2>System Requirements</h2><p>
EclEmma 2.0 will raise the bar a little bit and has the following system
requirements:
</p><ul>
<li>
<b>Java Runtime:</b> As JaCoCo requires Java 1.5 the same minimum JRE is
required for the Eclipse instance running EclEmma and the application under
test. Note that this does not apply to the Java class files under test: Any
class file version ranging from Java 1.0 to 1.7 is possible.
</li>
<li>
<b>Eclipse:</b> The minimum Eclipse Version to install EclEmma 2.0 is not
determined yet but the backward compatibility down to Eclipse 3.1 will be
dropped to get rid of several switches and usages of APIs deprecated in the
meantime. Anyhow a conservative approach is targeted, e.g. Eclipse 3.5 as the
minimal supported version.
</li>
<li>
<b>Operating System:</b> As JaCoCo as well as EclEmma is pure Java all
operating systems supported by Eclipse will also be supported including
Windows, Linux, Solaris, HP-UX, AIX and Mac OSX on different architectures.
</li>
</ul><h2>Future support for EMMA and EclEmma 1.x Maintenance</h2><p>
From EclEmma 2.0 on EMMA will not be supported any more as a coverage engine.
The only supported backend will be JaCoCo. There are not plans for a
"plugable" architecture to facilitate different coverage engines. The
technologies are simply too different to efficiently integrate them based on a
common framework.
</p><p>
Beside this we will try to maintain the EMMA based 1.x stream on a best effort
base but with no functional enhancements planned. There will be a separate
download for the 1.x versions.
</p><h2>Various Implementation Considerations</h2><p>
For a new EclEmma 2.x the following ideas should be considered:
</p><ul>
<li><b>API Compatibility:</b> As JaCoCo was explicitly designed for
integration, EclEmma 2.0 should be far less complex than the existing
1.x stream. To allow maximum reduction in code size and complexity several
of the existing APIs can either be simplified or removed at all.</li>
<li><b>Code Coverage Model:</b> Instead if defining a separate code coverage
model for Eclipse IDE usage only the JaCoCo model will be used directly.</li>
<li><b>Agent Communication:</b> Instead of writing Execution data to the file
system, the TCP/IP base communication will be used. This will also allow
intermediate dumps.</li>
<li><b>Increase Testability and Test Coverage:</b> Cutting several static
references to create more testable units will allow to increase unit test
coverage of EclEmma itself.</li>
</ul><h2>References</h2><ul>
<li>
EMMA - a free Java code coverage tool, Vlad Roubtsov, 2006<br />
<a class="extern" href="http://emma.sourceforge.net/">http://emma.sourceforge.net/</a>
</li>
<li>
EclEmma - Java Code Coverage for Eclipse, Marc R. Hoffmann et al., 2011<br />
<a href="../index.html">http://www.eclemma.org/</a>
</li>
<li>
JaCoCo - Java Code Coverage Library, Marc R. Hoffmann et al., 2011<br />
<a href="http://www.jacoco.org/jacoco">http://www.jacoco.org/jacoco</a>
</li>
<li>
Eclipse Marketplace, Eclipse Foundation, 2011<br />
<a class="extern" href="http://marketplace.eclipse.org/">http://marketplace.eclipse.org/</a>
</li>
<li>
JaCoCo Mission Statement, Marc R. Hoffmann, 2010<br />
<a class="extern" href="http://www.jacoco.org/jacoco/trunk/doc/mission.html">http://www.jacoco.org/jacoco/trunk/doc/mission.html</a>
</li>
<li>
JaCoCo Sonar Plugin, Evgeny Mandrikov et al., 2011<br />
<a class="extern" href="http://docs.codehaus.org/display/SONAR/JaCoCo+Plugin">http://docs.codehaus.org/display/SONAR/JaCoCo+Plugin</a>
</li>
</ul>
</td></tr>
</table>
<table cellspacing="0" cellpadding="0" border="0" style="width:100%;">
<tr>
<td class="footer"><a href="../license.html">Copyright</a> © 2006, 2017 Mountainminds GmbH & Co. KG and Contributors</td>
<td class="footer" style="text-align:right;">
Validate
<a href="http://validator.w3.org/check?uri=http://www.eclemma.org/devdoc/eclemma20.html">XHTML</a>/<a href="http://jigsaw.w3.org/css-validator/validator?uri=http://www.eclemma.org/devdoc/eclemma20.html">CSS</a> ·
Last Modified
<a href="https://github.com/jacoco/www.eclemma.org/blob/1ce487ba21186cba5d9a6195c273ae5557ec6e50/content/devdoc/eclemma20.html">2017-03-28</a> in Commit
<a href="https://github.com/jacoco/www.eclemma.org/commit/1ce487ba21186cba5d9a6195c273ae5557ec6e50">1ce487ba</a>
</td>
</tr>
</table>
</body>
</html>