i-nth logo


Gregg Rothermel, Lixin Li, Christopher DuPuis, & Margaret Burnett


Form-based visual programming languages, which include commercial spreadsheets and various research systems, have had a substantial impact on end-user computing.

Research shows, however, that form-based visual programs often contain faults. We would like to provide at least some of the benefits of formal testing methodologies to the creators of these programs.

This paper presents a testing methodology for form-based visual programs. To accommodate the evaluation model used with these programs, and the interactive process by which they are created, our methodology is validation-driven and incremental.

To accommodate the users of these languages, we provide an interface to the methodology that does not require an understanding of testing theory. We discuss our implementation of this methodology and empirical results achieved in its use.


Faults detected by the du-adequate test suites
Faults detected by the du-adequate test suites

This figure shows, for each program, the percentage of faults detected by the du-adequate test suites.

The overall average percentage of faults detected for all programs, faulty versions, and test suites in our study was 81%.

Fault detection varied across programs, but in all but one case (on two versions of Timecard) exceeded 50%.


1998, 20th International Conference on Software Engineering, April, pages 198-207

Full article

What You See Is What You Test