i-nth logo


Robin Abraham & Martin Erwig


Based on 1) research into mutation testing for general-purpose programming languages and 2) spreadsheet errors that have been reported in the literature, we have developed a suite of mutation operators for spreadsheets. We present an evaluation of the mutation adequacy of definition-use adequate test suites generated by a constraint-based automatic test-case generation system we have developed in previous work.

The results of the evaluation suggest additional constraints that can be incorporated into the system to target mutation adequacy. In addition to being useful in mutation testing of spreadsheets, the operators can be used in the evaluation of error-detection tools and also for seeding spreadsheets with errors for empirical studies.

We describe two case studies where the suite of mutation operators helped us carry out such empirical evaluations. The main contribution of this paper is a suite of mutation operators for spreadsheets that can be used for performing empirical evaluations of spreadsheet tools to indicate ways in which the tools can be improved.


Process for evaluating change suggestions
Process for evaluating change suggestions

We start with the set of spreadsheets and use the mutation operators to generate a (much larger) set of mutant spreadsheets. These are compared with the original spreadsheet to verify if the effect of the mutation was reversed by the application of any one of the generated change suggestions.


2009, IEEE Transactions on Software Engineering, Volume 35, Number 1, pages 94-108

Full article

Mutation operators for spreadsheets