pdf file (146 KB)John DeVale, Philip Koopman & David Guttendorf
ECE Department & Institute for Complex Engineered Systems
Carnegie Mellon University, Pittsburgh, Pennsylvania, USA
Published in the Proceedings of TCS’99, Washington DC.
Computers are becoming essential to everyday life in modern society, but are not necessarily as dependable as one would like, especially with respect to software robustness. Cost and time constraints often limit testing to the important area of functional correctness, but leave few resources to determine a software system’s robustness in the face of exceptional conditions. This paper describes the Ballista Project – a methodology and Web server that remotely test software modules in linkable object code form. By focusing on module interfaces rather than functional specifications, we have been able to build a scalable robustness evaluation framework having a test development cost sub-linear with respect to the number of modules to be tested. A client-server framework exercises the module under test on the developer’s computer. The Ballista server remotely directs testing, eliminating the need to port server or application code, and maintaining confidentiality of the source code under test. Stand-alone tests to replicate failures are generated and viewed on the World Wide Web. The Ballista server can be used to create or augment test suites, and to collect data to better understand the exceptional conditions that induce software robustness failures.
Paper:
pdf file (146 KB)Slides from conference presentation:
pdf file(709 KB)For results of testing Microsoft Windows, see:
Shelton, C. & Koopman, P., "Robustness Testing
of the Microsoft Win32 API, Dependable Systems and Networks Conference
2000/FTCS-30, New York City, June 26-28 2000.
