Abstract
While existing work concentrates on developing QoS models of business workflows and Web services, few tools have been developed to support the monitoring and performance analysis of scientific workflows in Grids. This paper describes novel Grid services for dynamic instrumentation of Grid-based applications, performance monitoring and analysis of Grid scientific workflows. We describe a Grid dynamic instrumentation service that provides a widely accessible interface for other services and users to conduct the dynamic instrumentation of Grid applications during the runtime. We introduce a Grid performance analysis service for Grid scientific workflows. The analysis service utilizes various types of data including workflow graphs, monitoring data of resources, execution status of activities, and performance measurements obtained from the dynamic instrumentation of invoked applications, and provides a rich set of functionalities and features to support the online monitoring and performance analysis of scientific workflows. Workflows and their relevant information including performance metrics are stored and utilized for comparing the performance of constructs of different workflows and for supporting multi-workflow analysis.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
W.M.P.V.D. Aalst, A.H.M.T. Hofstede, B. Kiepuszewski and A.P. Barros, ???Workflow Patterns???, Distrib. Parallel Databases, Vol. 14, No. 1, pp. 5???51, 2003.
A.F. Abate, A. Esposito, N. Grieco and G. Nota, ???Workflow Performance Evaluation through WPQL???, in Proceedings of the 14th International Conference on Software Engineering and Knowledge Engineering, 2002, pp. 489???495.
G. Allen, K. Davis, T. Goodale, A. Hutanu, H. Kaiser, T. Kielmann, A. Merzky, R. van Nieuwpoort, A. Reinefeld, F. Schintke, T. Sch??tt, E. Seidel and B. Ullmer, ???The Grid Application Toolkit: Towards Generic and Easy Application Programming Interfaces for the Grid???, Vol. 93, No. 3, pp. 534???550, 2005.
AustrianGrid, http://www.austriangrid.at/.
Z. Balaton and G. Gombas, ???Resource and Job Monitoring in the Grid???, in Proceedings of Euro-Par'2003 Parallel Processings, Klagenfurt, Austria, 2003.
Z. Balaton, P. Kacsuk, N. Podhorszki and F. Vajda, ???From Cluster Monitoring to Grid Monitoring Based on GRM???, in Proceedings of 7th EuroPar'2001 Parallel Processings, Manchester, UK, 2001, pp. 874???881.
B. Balis, M. Bubak, W. Funika, T. Szepieniec and R. Wism??ller, ???An Infrastructure for Grid Application Monitoring???, LNCS, Vol. 2474, pp. 41???49, 2002.
B. Balis, M. Bubak, W. Funika, R. Wism??ller, M. Radecki, T. Szepieniec, T. Arodz and M. Kurdziel, ???Grid Environment for On-line Application Monitoring and Performance Analysis???, Scientific Programming, Vol. 12, No. 4, pp. 239???251, 2004.
B. Buck and J.K. Hollingsworth, ???An API for Runtime Code Patching???, The International Journal of High Performance Computing Applications, Vol. 14, No. 4, pp. 317???329, 2000.
J. Cardoso, A.P. Sheth and J. Miller, ???Workflow Quality of Service???, in Proceedings of the IFIP TC5/WG5.12 International Conference on Enterprise Integration and Modeling Technique, 2003, pp. 303???311.
A. Cooke et al., ???R-GMA: An Information Integration System for Grid Monitoring???, in Proceedings of 11th International Conference on Cooperative Information Systems (CoopIS' 2003), Sicily, Italy, 2003.
E. Deelman, J. Blythe, Y. Gil, C. Kesselman, G. Mehta, K. Vahi, K. Blackburn, A. Lazzarini, A. Arbree and S. Koranda, ???Mapping Abstract Complex Workflows onto Grid Environments???, Journal of Grid Computing, Vol. 1, pp. 25???39, 2003.
L. DeRose, T. Hoover, Jr. and J. Hollingsworth, ???The Dynamic Probe Class Library: An Infrastucture for Developing Instrumentation for Performance Tools???, in Proceedings of the 15th International Parallel and Distributed Processing Symposium (IPDPS-01), Los Alamitos, CA, 2001, pp. 66???66.
T. Fahringer, A. Jugravu, S. Pllana, R. Prodan, C.S. Junior and H.-L. Truong, ???ASKALON: A Tool Set for Cluster and Grid Computing???, Concurrency and Computation: Practice and Experience, Vol. 17, Nos. 2???4, pp. 143???169, 2005.
I. Foster and C. Kesselman (eds.), The Grid: Blueprint for a New Computing Infrastructure. Morgan Kaufmann: San Francisco, CA, 1999.
M. Gerndt, R. Wismueller, Z. Balaton, G. Gombas, P. Kacsuk, Z. Nemeth, N. Podhorszki, H.-L. Truong, T. Fahringer, M. Bubak, E. Laure and T. Margalef, Performance Tools for the Grid: State of the Art and Future, Vol. 30 of Research Report Series, Lehrstuhl f??r Rechnertechnik und Rechnerorganisation (LRR-TUM), Technische Universit??t M??nchen, Shaker Verlag, 2004.
Globus Project, http://www.globus.org.
gSOAP: C/C++ Web Services and Clients, http://www.cs.fsu.edu/~engelen/soap.html.
D. Gunter, B. Tierney, B. Crowley, M. Holding and J. Lee, ???NetLogger: A Toolkit for Distributed System Performance Analysis???, in Proceedings of the IEEE Mascots 2000 Conference, 2000.
http://www-unix.globus.org/toolkit/docs/3.2/core/developer/message_security.html.
JFreeChart, http://www.jfree.org/jfreechart/.
JGraph, http://www.jgraph.com/.
P. Kacsuk, G. Dozsa, J. Kovacs, R. Lovas, N. Podhorszki, Z. Balaton and G. Gombas, ???P-GRADE: A Grid Programming Environment???, Journal of Grid Computing, Vol. 1, No. 2, pp. 171???197, 2003.
K.-H. Kim and C.A. Ellis, ???Performance Analytic Models and Analyses for Workflow Architectures???, Information Systems Frontiers, Vol. 3, No. 3, pp. 339???355, 2001.
S. Krishnan, P. Wagstrom and G. von Laszewski, ???GSFL: A Workflow Framework for Grid Services???, Technical report, Argonne National Laboratory, Argonne, IL, U.S.A., 2002.
???K-WF Grid Project, http://www.kwfgrid.net???.
G. Laszewski, I. Foster, J. Gawor and P. Lane, ???A Java Commodity Grid Kit???, Concurrency and Computation: Practice and Experience, Vol. 13, pp. 643???662, 2001.
B. Ludaescher, I. Altintas, C. Berkley, D. Higgins, E. Jaeger-Frank, M. Jones, E. Lee, J. Tao and Y. Zhao, ???Scientific Workflow Management and the Kepler System???, in Concurrency and Computation: Practice and Experience, Special Issue on Scientific Workflows, 2005.
B. Miller, M. Callaghan, J. Cargille, J. Hollingsworth, R. Irvin, K. Karavanic, K. Kunchithapadam and T. Newhall, ???The Paradyn Parallel Performance Measurement Tool???, IEEE Computer, Vol. 28, No. 11, pp. 37???46, 1995.
Montage, http://montage.ipac.caltech.edu.
Paradyn Parallel Performance Tools, http://www.cs.wisc.edu/paradyn/.
N. Podhorszki and P. Kacsuk, ???Monitoring Message Passing Applications in the Grid with GRM and R-GMA???, in Proceedings of EuroPVM/MPI'2003, Venice, Italy, 2003.
B.T.R. Savarimuthu, M. Purvis and M. Fleurke, ???Monitoring and Controlling of a Multi-Agent Based Workflow System???, in Proceedings of the 2nd Workshop on Australasian Information Security, Data Mining and Web Intelligence, and Software Internationalisation, 2004, pp. 127???132.
C. Seragiotto, T. Li, T. Fahringer, B. Mohr, M. Gerndt and H.-L. Truong, ???Standardized Interfaces for Representing, Instrumenting, and Monitoring Fortran, Java, C, and C++ Programs???, Concurrency and Computation: Practice and Experience, On submission, 2005.
M.P. Singh and M.A. Vouk, ???Scientific Workflows???, in Position Paper in Reference Papers of the NSF Workshop on Workflow and Process Automation in Information Systems: State-of-the-Art and Future Directions, 1996.
J.F. Sowa, Knowledge Representation: Logical, Philosophical, and Compuational Foundations. Brooks/Cole: Pacific Grove, CA, 2000.
The Condor Team, DAGMan (Directed Acyclic Graph Manager), http://www.cs.wisc.edu/condor/dagman/.
The Grid Resource Allocation and Management (GRAM), http://wwwunix.globus.org/toolkit/docs/3.2/gram/ws/index.html.
H.-L. Truong and T. Fahringer, ???SCALEA: A Performance Analysis Tool for Parallel Programs???, Concurrency and Computation: Practice and Experience, Vol. 15, Nos. 11???12, pp. 1001???1025, 2003.
H.-L. Truong and T. Fahringer, ???SCALEA-G: A Unified Monitoring and Performance Analysis System for the Grid???, Scientific Programming, Vol. 12, No. 4, pp. 225???237, 2004.
M.A. Vouk and M.P. Singh, ???Quality of Service and Scientific Workflows???, Technical report TR-96-19, Department of Computer Science, North Carolina State University, Thu., 19 Sep. 96 22:59:36 GMT, 1996.
J. Wainer, M. Weske, G. Vossen and C.B. Medeiros, ???Scientific Workflow Systems???, in Proceedings of NSF Workshop on Workflow and Process Automation in Information Systems: State-of-the-Art and Future Directions, Athens, Georgia, 1996.
V. Welch, F. Siebenlist, I. Foster, J. Bresnahan, K. Czajkowski, J. Gawor, C. Kesselman, S. Meder, L. Pearlman and S. Tuecke, ???Security for Grid Services???, in 12th IEEE International Symposium on High Performance Distributed Computing (HPDC'03), Seattle, Washington, 2003, pp. 48???57.
???Worldflow Management Coalition: Terminology and Glossary???, Technical report WFMC-TC-1011, February 1999.
S. Zanikolas and R. Sakellariou, ???A Taxonomy of Grid Monitoring Systems???, Future Generation Computing Systems, Vol. 21, No. 1, pp. 163???188, 2005.
Author information
Authors and Affiliations
Corresponding author
Additional information
The work described in this paper is supported in part by the Austrian Science Fund as part of the Aurora Project under contract SFBF1104 and by the European Union through the IST-2002-511385 project K-WfGrid.
Rights and permissions
About this article
Cite this article
Truong, HL., Fahringer, T. & Dustdar, S. Dynamic Instrumentation, Performance Monitoring and Analysis of Grid Scientific Workflows. J Grid Computing 3, 1–18 (2005). https://doi.org/10.1007/s10723-005-5299-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10723-005-5299-6