Authors
Raymond R. Panko
Abstract
Undergraduate MS business students developed and debugged a spreadsheet model from a word problem. This model consisted of a bid to build a wall. The problem was designed to be relatively simple and domain-free, to address the concern that past spreadsheet experiments may have used problems that were too difficult or that required domain knowledge that subjects did not have.
During the development phase, 72 subjects created the spreadsheet model. Even with this rather simple problem, 38% of the models contained an error.
This high number of incorrect spreadsheets was not due to subjects making many errors. They only made 0.4 errors per spreadsheet. In addition, their cell error rate (CER) was only 1.7%, meaning that only 1.7% of their cells contained errors. Unfortunately, spreadsheets tend to have long cascades of cells leading to the bottom line. This means that even tiny cell error rates will multiply into high rates of bottom-line errors.
In a debugging phase, subjects tried to debug their own models. Of 19 subjects with incorrect models who did the debugging part of the experiment, only three (16%) found and corrected their errors. So even with a relatively simple model, development and debugging were problematic.
This is a lower rate of finding errors than Galletta, et al. [1993, 1996] found when subjects debugged models created by the experimenter. This may mean that people are not as good at debugging their own models as they are at debugging models created by others.
Sample
Development phase:
| Number of subjects | 72 |
| Spreadsheets with errors, number | 27 |
| Spreadsheets with errors, percent | 38% |
| Errors per spreadsheet | 0.4 |
| Total errors | 30 |
| Omission errors | 17 |
| Logic errors | 13 |
| Mechanical errors | 0 |
| CER (cell error rate) overall | 1.7% |
| CER for omission errors | 0.9% |
| CER for logic errors | 1.4% |
Debugging phase:
| Debugged spreadsheets with errors | 19 |
| Correctly fixed, number | 3 |
| Correctly fixes, percentage | 16% |
Publication
1996, 29th Annual Hawaii International Conference on System Sciences, January, pages 356-363
Full article
Hitting the wall: Errors in developing 'simple' spreadsheet models
