Proactive detection of inadequate diagnostic messages for software configuration errors

Download: PDF, slides (PDF), slides (PowerPoint), ConfDiagDetector implementation.

“Proactive detection of inadequate diagnostic messages for software configuration errors” by Sai Zhang and Michael D. Ernst. In ISSTA 2015, Proceedings of the 2015 International Symposium on Software Testing and Analysis, (Baltimore, MD, USA), July 2015, pp. 12-23.

Abstract

This paper presents a technique to detect inadequate (i.e., missing or ambiguous) diagnostic messages for configuration errors issued by a configurable software system.

The technique injects configuration errors into the software under test, monitors the software outcomes under the injected configuration errors, and uses natural language processing to analyze the output diagnostic message caused by each configuration error. The technique reports diagnostic messages that may be unhelpful in diagnosing a configuration error.

We implemented the technique for Java in a tool, ConfDiagDetector. In an evaluation on 4 real-world, mature configurable systems, ConfDiagDetector reported 43 distinct inadequate diagnostic messages (25 missing and 18 ambiguous). 30 of the detected messages have been confirmed by their developers, and 12 more have been identified as inadequate by users in a user study. On average, ConfDiagDetector required 5 minutes of programmer time and 3 minutes of compute time to detect each inadequate diagnostic message.

Download: PDF, slides (PDF), slides (PowerPoint), ConfDiagDetector implementation.

BibTeX entry:

@inproceedings{ZhangE2015,
   author = {Sai Zhang and Michael D. Ernst},
   title = {Proactive detection of inadequate diagnostic messages for
	software configuration errors},
   booktitle = {ISSTA 2015, Proceedings of the 2015 International
	Symposium on Software Testing and Analysis},
   pages = {12--23},
   address = {Baltimore, MD, USA},
   month = jul,
   year = {2015}
}

(This webpage was created with bibtex2web.)

Back to Michael Ernst's publications.