Data Analysis Using N-Version Voting

• Use N-version software voting to refine data(and use manual sampling to check effectiveness)

  • Eliminate non-exceptional tests -- 12% of data; method ~100% accurate
    • e.g., reading from read-only file
  • Identify Silent failures

• Silent failures -- 6% to 17% additional robustness failure rate

  • 80% accurate when one OS reports “OK” while at least one other OS reports an error code
    • ~2% were bugs involving failure to write past end of file
    • 28% of remainder due when POSIX permits either case
    • 30% of remainder due to false alarm error codes (many in QNX)
    • ~40% of remainder just out of scope of POSIX standard
  • 50% accurate when one OS reports “OK” but another OS dumps core
    • Half of remainder due to order in which parameters are checked
    • Half of remainder due to FreeBSD floating point libraryAbort failures (e.g., fabs(DBL_MAX) )

Previous slide

Next slide

Back to first slide

View graphic version