Spreadsheets can be viewed as a highly flexible programming environment for end users.
Abreu, et al (2015)
Most executives do not really check or verify the accuracy or validity of [their] spreadsheets...
Teo & Tan (1999)
Errors in spreadsheets... result in incorrect decisions being made and significant losses incurred.
Beaman, et al (2005)
Despite overwhelming and unanimous evidence... companies have continued to ignore spreadsheet error risks.
Panko (2014)
Studies have shown that there is a high incidence of errors in spreadsheets.
Csernoch & Biro (2013)
Spreadsheets are often hard, if not impossible, to understand.
Mireault & Gresham (2015)
Spreadsheet errors have resulted in huge financial losses.
Abraham & Erwig (2007)
...few incidents of spreadsheet errors are made public and these are usually not revealed by choice.
Kruck & Sheetz (2001)
Spreadsheets... pose a greater threat to your business than almost anything you can imagine.
Howard (2005)
The software that end users are creating... is riddled with errors.
Burnett & Myers (2014)
People tend to believe their spreadsheets are more accurate than they really are.
Caulkins, Morrison, & Weidemann (2006)
The quality and reliability of spreadsheets is known to be poor.
Bishop & McDaid (2007)
Programmers exhibit unwarranted confidence in the correctness of their spreadsheets.
Krishna, et al (2001)
Despite being staggeringly error prone, spreadsheets are a highly flexible programming environment.
Abreu, et al (2015)
94% of the 88 spreadsheets audited in 7 studies have contained errors.
Panko (2008)
Spreadsheets are the most popular live programming environments, but they are also notoriously fault-prone.
Hermans & van der Storm (2015)
Research on spreadsheet errors is substantial, compelling, and unanimous.
Panko (2015)
It is now widely accepted that errors in spreadsheets are both common and potentially dangerous.
Nixon & O'Hara (2010)
Spreadsheets are commonly used and commonly flawed.
Caulkins, Morrison, & Weidemann (2008)
Spreadsheet shortcomings can significantly hamper an organization's business operation.
Reschenhofer & Matthes (2015)
Every study, without exception, has found error rates much higher than organizations would wish to tolerate.
Panko (1999)
Spreadsheets are notoriously error-prone.
Cunha, et al (2011)
Spreadsheets are extraordinarily and unacceptably prone to error.
Dunn (2010)
Spreadsheet errors... a great, often unrecognised, risk to corporate decision making & financial integrity.
Chadwick (2002)
Even obvious, elementary errors in very simple, clearly documented spreadsheets are... difficult to find.
Galletta, et al (1993)
The results given by spreadsheets are often just wrong.
Sajaniemi (1998)
The issue is not whether there is an error but how many errors there are and how serious they are.
Panko (2007)
Every study that has looked for errors has found them... in considerable abundance.
Panko & Halverson (1996)
Developing an error-free spreadsheet has been a problem since the beginning of end-user computing.
Mireault (2015)
The untested spreadsheet is as dangerous and untrustworthy as an untested program.
Price (2006)
Spreadsheet errors are still the rule rather than the exception.
Nixon & O'Hara (2010)
Spreadsheets are easy to use and very hard to check.
Chen & Chan (2000)
60% of large companies feel 'Spreadsheet Hell' describes their reliance on spreadsheets.
Murphy (2007)
Spreadsheets are alarmingly error-prone to write.
Paine (2001)
1% of all formulas in operational spreadsheets are in error.
Powell, Baker, & Lawson (2009)
Overconfidence is one of the most substantial causes of spreadsheet errors.
Sakal, et al (2015)
A significant proportion of spreadsheets have severe quality problems.
Ayalew (2007)
Untested spreadsheets are riddled with errors.
Miller (2005)
A lot of decisions are being made on the basis of some bad numbers.
Ross (1996)
Spreadsheet development must embrace extensive testing in order to be taken seriously as a profession.
Bock (2016)
Spreadsheets are dangerous to their authors and others.
Durusau & Hunting (2015)
Errors in spreadsheets are as ubiquitous as spreadsheets themselves.
Colbenz (2005)
Spreadsheets are more fault-prone than other software.
Kulesz & Ostberg (2013)
Spreadsheets contain errors at an alarmingly high rate.
Abraham, et al (2005)
Spreadsheet errors are pervasive, stubborn, ubiquitous and complex.
Irons (2003)
Your spreadsheets may be disasters in the making.
Caulkins, Morrison, & Weidemann (2006)
It is irrational to expect large error-free spreadsheets.
Panko (2013)
Spreadsheets have a notoriously high number of faults.
Rust, et al (2006)
Most large spreadsheets have dozens or even hundreds of errors.
Panko & Ordway (2005)
Never assume a spreadsheet is right, even your own.
Raffensperger (2001)

i spreadsheet testing methodology - Overview

In this series of articles we introduce the i methodology for testing spreadsheets.

The i spreadsheet testing methodology is a set of guidelines for validating that a spreadsheet does what it is intended to do and ensuring that it will continue to do so.

The i methodology consists of five components, each of which asks five questions:

  • Intent. Is the spreadsheet fit-for-purpose? Is the spreadsheet solving the right problem? Do stakeholders understand the intent of the spreadsheet? Does the spreadsheet answer all the necessary questions? Is the scope limited to only the required tasks?
  • Instructions. Are there clear instructions about how to use the spreadsheet? Does the spreadsheet include instructions for developers? Are all the instructions up-to-date? Are all data sources defined? Does all data have units specified?
  • Instruments. Are the instruments (techniques, tools, and algorithms) appropriate? Would other instruments be better? Do stakeholders understand the spreadsheet's instruments? Do the developers understand the instruments? Are limitations and caveats clearly stated?
  • Implementation. Are the instruments implemented correctly? Is the design intuitive, easy to use, and robust? Is there clear separation between data and calculations? Does the spreadsheet have access and version control? Is recalculation time an issue?
  • Immunity. To what extent is the spreadsheet immune to errors? How well does the spreadsheet handle invalid data? Is the spreadsheet easy to modify without creating errors? Are the error messages adequate? Does the spreadsheet fail gracefully when necessary?
Overview of the i spreadsheet testing methodology
i-nth spreadsheet testing methodology
The i methodology is a comprehensive approach to spreadsheet testing, consisting of five parts: Intent, Instructions, Instruments, Implementation, and Immunity.

These questions have been distilled from our extensive experience in building, working with, and testing spreadsheets. We have also reviewed the spreadsheet error and testing literature for research into the causes of spreadsheet errors and proven practices that reduce the incidence and impact of errors.

Many of the questions in the i spreadsheet testing methodology, and the issues that they relate to, are simply common sense guidelines. However, it is remarkable how often these guidelines are not followed in practice.

The consequences of not following these guidelines range from trivial, annoying errors that result in productivity losses, through to the disastrous failure of an entire organisation. For some examples of the impacts that spreadsheet errors can have, see EuSpRIG horror stories.

A central theme underpinning the i spreadsheet testing methodology is that we are seeking to give the stakeholders of a spreadsheet confidence that it does what it should – starting with having the right intent, though to it being well designed, implemented correctly, used correctly and finally to it producing correct results. Every aspect of the methodology is a necessary condition for stakeholders to have confidence in their spreadsheets.

Each component of the i spreadsheet testing methodology is explained in the following pages.

 

Next: Intent

Go to top