Ballista® Publications		®
Carnegie Mellon University

Link to Ballista home page

One-page project overview (.pdf).

Presentations:

• Koopman, P., "Reliability, Safety, and Security in Everyday Embedded Systems," Latin American Dependability Conference (LADC-07), Morelia, Mexico, Sept. 26-28, 2007
• June 2002 Retrospective presentation slidesof what we learned on the Ballista project.
• Overview presentation slides of the Ballista project from late 2000 and
  a Real Video movie of a lecture.
• November 1999 "Shrink-wrapped" Unix Robustness Testing Suite slides, ISSRE '99
• March 1999 PI meeting poster
• June 1998 FTCS presentation slides
• March 1998 PI meeting poster
• March 1997 Portable OS robustness testing slides
• Quicktime Video of Ballista-identified test crashing Windows 98 | test crashing Windows ME

Publications:

• Koopman, P., DeVale, K. & DeVale, J., "Interface robustness testing: experiences and lessons learned from the Ballista Project," In: Kanoun, K. & Spainhower, L., Eds., Dependability Benchmarking for Computer Systems, IEEE Press, 2008, pp. 201-226.
• Robust software - no more excuses, DSN 2002, DSN 2002
• High Performance Robust Computer Systems (Ph.D. thesis, 2002)
• Workshop on Dependability Benchmarking, DSN 2002 (complete workshop proceedings available on-line)
• What’s Wrong With Fault Injection As A Benchmarking Tool?, Workshop on Dependability Benchmarking/DSN 2002
• Performance Evaluation of Exception Handling in I/O Libraries, DSN 2001
• Robustness testing and hardening of CORBA ORB Implementations, DSN 2001
• Dependability benchmarking: making choices in an n-dimensional problem space, EASY Workshop/DSN 2001
• The Exception Handling Effectiveness of POSIX Operating Systems, IEEE Transactions on Software Engineering, Sept. 2000.
• Robustness Testing of the Microsoft Win32 API, DSN 2000
• Robustness Testing of the Microsoft Win32 API (M.S. project report, 2000)
• Robustness Testing of a Distributed Simulation Backplane, ISSRE 99
• Comparing the Robustness of POSIX Operating Systems, FTCS '99
• The Ballista Software Robustness Testing Service, TCS '99
• A Dimensionality Model Approach to Testing and Improving Software Robustness, ATC'99
• The Dimensionality of Failures - A Fault Model for Characterizing Software Robustness, FTCS '99 student paper
• Testing Protocol Implementation Robustness, FTCS '99 student paper
• Dependability Benchmarking & Prediction: A Grand Challenge Technology Problem, The 1st International Workshop on Real-Time Mission-Critical Systems: Grand Challenge Problems, 1999
• Robustness Testing of A Distributed Simulation Backplane (M.S. project report, 1999)
• Toward a Scalable Method for Quantifying Aspects of Fault Tolerance, Software Assurance, and Computer Security CSDA '98
• Measuring Operating System Robustness (M.S. project report)
• Automated Robustness Testing of Off-the-Shelf Software Components, FTCS, June 1998
• Automatic Robustness Testing of Off-the-Shelf Software Components (M.S. project report, 1998)
• "Comparing Operating Systems using Robustness Benchmarks," 16th IEEE Symposium on Reliable Distributed Systems (SRDS97)
• Ballista Architecture & Methodology (technical report)
• Ballista precursor work: "Middleware Enabled Fault Management for Commercial Operating Systems" (M.S. thesis)

• PC Week column mention of Ballista (September, 1999)
• Military & Aerospace Electronics article on Ballista (March, 1998)

Related Web Sites:

• IFIP WG 10.4 SIG on Dependability Benchmarking (SIGDeB)
• Workshop on Dependability Benchmarking/DSN-2002

Ballista Frequently Asked Questions (FAQ)

• What is the goal of the Ballista project?
• What is robustness?
• What is a robustness failure?
• How does Ballista work?
• What is an example of the kind of robustness failures Ballista tests for?
• Are the things Ballista finds really bugs?
• What is the relationship between number of software defects and the number of robustness failures?
• Doesn't providing robustness cost speed?
• How does Ballista relate to other robustness testing?
• Isn't Ballista just the same as Purify or Boundschecker?
• How can I get access to Ballista testing?
• Are you related to the Ballista OS security product?

Discussion (e-mail from OS vendors, included with permission)

• QNX: 12 March 1998

• What the heck does "Ballista" mean, anyway?

"Ballista^®" is a registered trademark of Carnegie Mellon University.

Link to Ballista home page

Web site by Phil Koopman.
Comments to: koopman@cmu.edu