Using Dynatrace Monitoring Data for Generating

Performance Models of Java EE Applications

Felix Willnecker Wolfgang Gottesheim Helmut Krcmar

Andreas Brunnert Compuware Austria GmbH Technische Universität
fortiss GmbH Freistädter Str. 313 München
Guerickestr. 25 4040 Linz, Austria Boltzmannstr. 3
80805 München, Germany wolfgang.gottesheim 85748 Garching, Germany
{willnecker,brunnert} @dynatrace.com [email protected]
@fortiss.org

ABSTRACT lack of tool support for predicting the required capacity of

Performance models assist capacity management and plan- a software system for expected workloads [5]. Performance
ning for large-scale enterprise applications by predicting their models and corresponding model solvers or simulation en-
performance for different workloads and hardware environ- gines can enhance current capacity estimations and there-
ments. Manually creating these models often outweighs fore increase the utilization of hardware and reduce costs
their benefits. Automatic performance model generators for application operations [2, 7].
have been introduced to facilitate the model creation. These The effort of manually creating such performance models
generators often use custom monitoring solutions to generate often outweighs their benefits [8]. Automatic model genera-
the required input data for the model creation. In contrast, tors have been introduced to reduce this effort [4, 2]. These
standardized application performance management (APM) approaches rely on monitoring data from running systems
solutions are used in industry to control performance met- to extract the performance models. Such generated models
rics for productive systems. This work presents the integra- can be used as input for a simulation engine or an analytical
tion of industry standard APM solutions with a performance solver to predict the resource utilization, throughput and
model generation framework. We apply the integration con- response time for different workloads and hardware environ-
cepts using the APM solution Dynatrace and a performance ments.
model generation framework for Palladio Component Mod- Monitoring data for the generation of performance mod-
els (PCM). els is gathered by either custom solutions or tools from the
scientific community [4, 13]. On the other hand, monitoring
of large-scale EAs are state of the art technology in prac-
Categories and Subject Descriptors tice [9]. Companies use the gathered monitoring data to
C.4 [Performance of Systems]: measurement techniques, detect and resolve performance problems in productive en-
modeling techniques vironments [10]. This work presents an extension of our
existing performance model generation framework to work
General Terms with industry standard Application Performance Manage-
ment (APM) solutions. We extend the Performance Man-
Measurement, Performance, Prediction agement Work Tools (PMWT1 ) model generator to create
Palladio Component Models (PCM) based on data collected
Keywords by the Dynatrace2 APM solution [1, 12, 6, 4].
Load Testing; Performance Evaluation; Application Perfor-
mance Management 2. AUTOMATIC PERFORMANCE MODEL
GENERATION FRAMEWORK
1. INTRODUCTION In order to use the Dynatrace APM solution we extend
Performance of large-scale enterprise applications (EA) is the model generation framework presented in [4] and shown
a critical quality requirement [3]. Application providers and in figure 1. This framework uses a custom agent that col-
data center operators tend to over-provision capacity to en- lects the monitoring data from a running Java EE applica-
sure that performance goals are met [11]. This is due to a tion. The monitoring data is then processed and aggregated
Permission to make digital or hard copies of part or all of this work for personal or either as comma-separated value (CSV) files and imported
classroom use is granted without fee provided that copies are not made or distributed into a database or as Managed Beans (MBeans). The aggre-
for profit or commercial advantage and that copies bear this notice and the full citation gated data is input for the model generation. The result is
on the first page. Copyrights for third-party components of this work must be honored. a performance model compliant with the PCM meta-model.
For all other uses, contact the Owner/Author. The extension proposed in this work allows to use data ex-
tracted by standard monitoring frameworks exemplified by
Copyright is held by the owner/author(s).
a Dynatrace agent for the purpose of performance model
ICPE’15, Jan. 31–Feb. 4, 2015, Austin, TX, USA. 1
ACM 978-1-4503-3248-4/15/01.
http://pmw.fortiss.org/
2
http://dx.doi.org/10.1145/2668930.2688061. http://www.dynatrace.com/
 4. REFERENCES

Performance Model Generator

PMWT PMWT

Monitoring Data Persistence

CSV
Agent CSV Connector
CSV [1] S. Becker, H. Koziolek, and R. Reussner. The palladio
Java EE Application

Monitoring component model for model-driven performance

Service
Database
MBeans
Performance prediction. Journal of Systems and Software,
Model
82(1):3–22, 2009. Special Issue: Software Performance
Dynatrace Dynatrace Dynatrace - Modeling and Analysis.
Agent Server Connector
[2] F. Brosig, N. Huber, and S. Kounev. Modeling
parameter and context dependencies in online
Dynatrace
architecture-level performance models. In Proceedings
Session
Performance of the 15th ACM SIGSOFT Symposium on
Warehouse
Store Component Based Software Engineering, CBSE ’12,
pages 3–12, New York, NY, USA, 2012. ACM.
Figure 1: PMWT Performance Model Generation [3] A. Brunnert, C. Vögele, A. Danciu, M. Pfaff,
Framework M. Mayer, and H. Krcmar. Performance management
work. Business & Information Systems Engineering,
6(3):177–179, 2014.
generation. This agent is attached using runtime options [4] A. Brunnert, C. Vögele, and H. Krcmar. Automatic
without changes to the instrumented application system’s performance model generation for java enterprise
source code. The agent forwards collected data to the Dyna- edition (ee) applications. In M. Balsamo,
trace server, where detailed traces about method calls and W. Knottenbelt, and A. Marin, editors, Computer
error states are stored in session files for further analysis. Performance Engineering, 10th European Workshop
Performance metrics derived from these traces are stored in on Performance Engineering, volume 8168 of Lecture
a performance warehouse, and these metrics are typically Notes in Computer Science, pages 74–88. Springer
used by operation engineers as data provider for monitoring Berlin Heidelberg, 2013.
dashboards. We extract data from both sources using an ex- [5] A. Brunnert, K. Wischer, and H. Krcmar. Using
tension to our model generation framework called Dynatrace architecture-level performance models as resource
connector. profiles for enterprise applications. In Proceedings of
The Dynatrace connector leverages the representational the 10th International ACM Sigsoft Conference on
state transfer (REST) interface of the Dynatrace server to Quality of Software Architectures, QoSA ’14, pages
extract detailed monitoring data. This REST interfaces pro- 53–62, New York, NY, USA, 2014. ACM.
vides, among others, call traces for instrumented operations [6] B. Greifeneder. Method and system for processing
including their resource demands. The Dynatrace connec- application performance data ouside of monitored
tor is an extension of the monitoring data persistence service applications to limit overhead caused by monitoring,
that is used by the model generator to access data from dif- June 2011. US Patent 7,957,934.
ferent sources. The model generator creates a performance [7] L. Grinshpan. Solving Enterprise Applications
model conforming to the PCM meta-model based on the Performance Puzzles: Queuing Models to the Rescue.
traces and their average resource demands. The resulting John Wiley & Sons, 1st edition, 2012.
models can then be used for the existing simulation engines [8] S. Kounev. Performance modeling and evaluation of
and analytical solvers that exist for PCM models [12]. distributed component-based systems using queueing
petri nets. IEEE Transactions on Software
3. CONCLUSION & FUTURE WORK Engineering, 32(7):486–502, 2006.
[9] J. Kowall and W. Cappelli. Magic quadrant for
This work proposed an integration of an industry APM
application performance monitoring. Gartner Research
solution with a performance model generation framework.
ID G, 232180, 2012.
Different input formats and levels of granularity can be pro-
[10] H. Koziolek. Performance evaluation of
cessed. The extension shows that the generator and its in-
component-based software systems: A survey.
terface are generally applicable and other APM solutions as
Performance Evaluation, 67(8):634 – 658, 2010.
generator input are possible. As the Dynatrace solution is
Special Issue on Software and Performance.
in widespread use, the monitoring technology is tested more
intensive than custom solutions and in varied operation en- [11] M. Pawlish, A. Varde, and S. Robila. Analyzing
vironments. The generated model can be used to simulate utilization rates in data centers for optimizing energy
different workloads and therefore enhance the Dynatrace so- management. In Green Computing Conference
lution with capacity planning capabilities. (IGCC), 2012 International, pages 1–6, June 2012.
As a next step we will further extend an existing prototype [12] R. Reussner, S. Becker, E. Burger, J. Happe,
for the integration and evaluate it in a case study compar- M. Hauck, A. Koziolek, H. Koziolek, K. Krogmann,
ing the results using our PMWT agent and the Dynatrace and M. Kuperberg. The palladio component model.
agent. For the evaluation, we will extract models from a Journal of Systems and Software, 82(1):3 – 22, 2009.
running SPECjEnterprise2010 instance using the two exist- [13] A. van Hoorn, J. Waller, and W. Hasselbring. Kieker:
ing data collection approaches. Afterwards, the resulting A framework for application performance monitoring
models are used to predict the utilization, throughput and and dynamic software analysis. In Proceedings of the
response time for an increased number of users. The pre- 3rd ACM/SPEC International Conference on
diction results are compared with measurement for similar Performance Engineering, ICPE ’12, pages 247–248,
workloads on the SPECjEnterprise2010 instance. New York, NY, USA, 2012. ACM.