iⁿ spreadsheet testing methodology — Overview

In this series of articles we introduce the iⁿ methodology for testing spreadsheets.

The iⁿ spreadsheet testing methodology is a set of guidelines for validating that a spreadsheet does what it is intended to do and ensuring that it will continue to do so.

The iⁿ methodology consists of five components, each of which asks five questions:

• Intent. Is the spreadsheet fit-for-purpose? Is the spreadsheet solving the right problem? Do stakeholders understand the intent of the spreadsheet? Does the spreadsheet answer all the necessary questions? Is the scope limited to only the required tasks?
• Instructions. Are there clear instructions about how to use the spreadsheet? Does the spreadsheet include instructions for developers? Are all the instructions up-to-date? Are all data sources defined? Does all data have units specified?
• Instruments. Are the instruments (techniques, tools, and algorithms) appropriate? Would other instruments be better? Do stakeholders understand the spreadsheet's instruments? Do the developers understand the instruments? Are limitations and caveats clearly stated?
• Implementation. Are the instruments implemented correctly? Is the design intuitive, easy to use, and robust? Is there clear separation between data and calculations? Does the spreadsheet have access and version control? Is recalculation time an issue?
• Immunity. To what extent is the spreadsheet immune to errors? How well does the spreadsheet handle invalid data? Is the spreadsheet easy to modify without creating errors? Are the error messages adequate? Does the spreadsheet fail gracefully when necessary?

Overview of the iⁿ spreadsheet testing methodology

The iⁿ methodology is a comprehensive approach to spreadsheet testing, consisting of five parts: Intent, Instructions, Instruments, Implementation, and Immunity.

These questions have been distilled from our extensive experience in building, working with, and testing spreadsheets. We have also reviewed the spreadsheet error and testing literature for research into the causes of spreadsheet errors and proven practices that reduce the incidence and impact of errors.

Many of the questions in the iⁿ spreadsheet testing methodology, and the issues that they relate to, are simply common sense guidelines. However, it is remarkable how often these guidelines are not followed in practice.

The consequences of not following these guidelines range from trivial, annoying errors that result in productivity losses, through to the disastrous failure of an entire organisation. For some examples of the impacts that spreadsheet errors can have, see EuSpRIG horror stories.

A central theme underpinning the iⁿ spreadsheet testing methodology is that we are seeking to give the stakeholders of a spreadsheet confidence that it does what it should — starting with having the right intent, through to it being well-designed, implemented correctly, used correctly and finally to it producing correct results. Every aspect of the methodology is a necessary condition for stakeholders to have confidence in their spreadsheets.

Each component of the iⁿ spreadsheet testing methodology is explained in the following pages.

Next: Intent