Scalable Test Generation -- An Oracle
Problem 2: Avoid per-test work to determine pass vs. fail
- Understanding of functionality required for “pass” vs. “fail”
- Insight: Ignore functionality -- use “doesn’t crash; doesn’t hang”
- Solution: Test them all and let the watchdog timer/core dumps sort them out.
- Test only Catastrophic - Restart - Abort failures (for now, anyway)
- Ignore pass/fail in terms of return code; just look for robustness failures
Example:
- read() succeeds but write() (hopefully) returns an error when accessing a read-only file
- But we can ignore any return codes and just look for an Abort or Restart in either case