Workshop on Dependability Benchmarking

In conjunction with: The International Conference on Dependable Systems and Networks, DSN-2002; June 25th, 2002, Washington, D.C., USA. http://www.dsn.org

Workshop Co-Chairs:
     Philip Koopman (Carnegie Mellon Univ., USA) koopman@cmu.edu
     Henrique Madeira (Univ. of Coimbra, Portugal) henrique@dei.uc.pt

• Proceedings

• Workshop Description
• Call for submissions (CLOSED)
• Program Committee

Also, see the web site for the IFIP WG 10.4 SIG on Dependability Benchmarking:
http://www.laas.fr/~kanoun/ifip_wg_10_4_sigdeb/index.htm

Workshop Proceedings:

Download a preprint of the entire proceedings: Adobe Acrobat format, 1.4 MB

Individual papers (Adobe Acrobat format):

• Preface, Philip Koopman & Henrique Madeira
• “Progress on Defining Standardized Classes for Comparing the Dependability of Computer Systems,” Don Wilson Compaq/Tandem Labs, USA; Brendan Murphy, Microsoft UK; Lisa Spainhower, IBM, USA
• “A Framework for Dependability Benchmarking,” Karama Kanoun, LAAS-CNRS, France; Henrique Madeira, University of Coimbra, Portugal; Jean Arlat, LAAS-CNRS, France
• “Including the Human Factor in Dependability Benchmarks,” Aaron B. Brown, Leonard C. Chung, and David A. Patterson, University of California at Berkeley, USA
• "Benchmarking Semantic Availability of Dynamic Data Feeds,” Orna Raz, Philip Koopman, and Mary Shaw, Carnegie Mellon University, USA
• “Using Bayesian Theory for Estimating Dependability Benchmark Measures,” Michel Cukier and Carol S. Smidts, University of Maryland at College Park, USA
• “Empirical Evaluation of Techniques and Methods Used for Achieving and Assessing Software High Dependability,” Ioana Rus, Fraunhofer Center for Empirical Software Engineering, USA; Victor Basili and Marvin Zelkowitz, University of Maryland USA; Barry Boehm, University of Southern California, USA
• “The Set-Check-Use Methodology for Detecting Error Propagation Failures in I/O Routines,” Michael W. Bigrigg and Jacob J. Vos, Carnegie Mellon University, USA
• “Defect and Fault Seeding In Dependability Benchmarking,” Barry Boehm and Daniel Port, University of Southern California, USA
• “System Recovery Benchmarking,” Ji Zhu, James Mauro, and Ira Pramanick, Sun Microsystems, USA
• “Faultload Representativeness for Dependability Benchmarking,” Jean Arlat and Yves Crouzet, LAAS-CNRS, Franc
• “What’s Wrong With Fault Injection As A Benchmarking Tool?” Philip Koopman, Carnegie Mellon University, USA

Workshop Description:

Classical features such as raw performance and functionality have long driven the computer industry to improve their products. But now, dependability and maintainability are seen as equally important. While there are relatively straightforward ways to evaluate and compare performance and functionality of different systems or components, the evaluation of dependability and maintainability features is much more difficult. Among the challenges that must be addressed are: incorporating the effects of software failures, characterizing the dependability of opaque off-the-shelf hardware and software components, including the effects of typical maintenance, operational, and configuration management procedures, and accommodating the fact that different application areas have different requirements for the various factors influencing dependability.

The goal of the Dependability Benchmarking Workshop is to provide a forum for the computer industry and academia to discuss problems associated with the evaluation and characterization of dependability and maintainability of components and computer systems. The identification of dependability benchmarking measures and the essential technologies for dependability benchmarking, including both experimental measuring and modeling technologies, are central aspects of this large discussion meant to garner ideas on practical and cost-effective ways to evaluate dependability and maintainability features.

Contributions are solicited in the form of short papers, work-in-progress reports, and position papers. Accepted workshop contributions (written material) will be published in the Supplement of the 2002 International Conference on Dependable Systems and Networks. The workshop will be held on one day during the DSN-2002 conference. Workshops are not alternative forums for full papers, which otherwise could be submitted to DSN. Workshop attendance is open to all registered conference attendees. We welcome participation by professionals with diverse backgrounds who can contribute to advancing the technology and understanding of the workshop subject.

Participants are invited to browse the on-line presentations of a previous meeting on this topic held in March 2001: 2001 Workshop. (That was a presentation-only workshop; the 2002 workshop will have published proceedings as mentioned above.)

Program Committee:
Koopman, Philip (CMU, USA) co-chair
Madeira, Henrique (Univ. Coimbra, Portugal) co-chair
Blanquart, Jean-Paul (Astrium, France)
Cukier, Michel (UIUC, USA)
Kanoun, Karama (LAAS-CNRS, France)
Karlsson, Johan (Chalmers Univ. Sweden)
Murphy, Brendan (Microsoft Research, UK)
Wilson, Don (Compaq, USA)

wdb@koopman.cc | Main Workshop Page