Even obvious, elementary errors in very simple, clearly documented spreadsheets are... difficult to find.
Galletta, et al (1993)
The untested spreadsheet is as dangerous and untrustworthy as an untested program.
Price (2006)
Your spreadsheets may be disasters in the making.
Caulkins, Morrison, & Weidemann (2006)
60% of large companies feel 'Spreadsheet Hell' describes their reliance on spreadsheets.
Murphy (2007)
Despite overwhelming and unanimous evidence... companies have continued to ignore spreadsheet error risks.
Panko (2014)
Spreadsheet shortcomings can significantly hamper an organization's business operation.
Reschenhofer & Matthes (2015)
Spreadsheets are alarmingly error-prone to write.
Paine (2001)
Spreadsheets are more fault-prone than other software.
Kulesz & Ostberg (2013)
1% of all formulas in operational spreadsheets are in error.
Powell, Baker, & Lawson (2009)
Spreadsheet errors are pervasive, stubborn, ubiquitous and complex.
Irons (2003)
Spreadsheets are commonly used and commonly flawed.
Caulkins, Morrison, & Weidemann (2008)
The quality and reliability of spreadsheets is known to be poor.
Bishop & McDaid (2007)
It is irrational to expect large error-free spreadsheets.
Panko (2013)
Most large spreadsheets have dozens or even hundreds of errors.
Panko & Ordway (2005)
The results given by spreadsheets are often just wrong.
Sajaniemi (1998)
Programmers exhibit unwarranted confidence in the correctness of their spreadsheets.
Krishna, et al (2001)
Research on spreadsheet errors is substantial, compelling, and unanimous.
Panko (2015)
Studies have shown that there is a high incidence of errors in spreadsheets.
Csernoch & Biro (2013)
Errors in spreadsheets are as ubiquitous as spreadsheets themselves.
Colbenz (2005)
Untested spreadsheets are riddled with errors.
Miller (2005)
Every study that has looked for errors has found them... in considerable abundance.
Panko & Halverson (1996)
Errors in spreadsheets... result in incorrect decisions being made and significant losses incurred.
Beaman, et al (2005)
Most executives do not really check or verify the accuracy or validity of [their] spreadsheets...
Teo & Tan (1999)
Never assume a spreadsheet is right, even your own.
Raffensperger (2001)
A lot of decisions are being made on the basis of some bad numbers.
Ross (1996)
94% of the 88 spreadsheets audited in 7 studies have contained errors.
Panko (2008)
Every study, without exception, has found error rates much higher than organizations would wish to tolerate.
Panko (1999)
Spreadsheets are dangerous to their authors and others.
Durusau & Hunting (2015)
Spreadsheet errors... a great, often unrecognised, risk to corporate decision making & financial integrity.
Chadwick (2002)
People tend to believe their spreadsheets are more accurate than they really are.
Caulkins, Morrison, & Weidemann (2006)
Despite being staggeringly error prone, spreadsheets are a highly flexible programming environment.
Abreu, et al (2015)
The issue is not whether there is an error but how many errors there are and how serious they are.
Panko (2007)
Spreadsheets can be viewed as a highly flexible programming environment for end users.
Abreu, et al (2015)
Spreadsheets are extraordinarily and unacceptably prone to error.
Dunn (2010)
Spreadsheets are often hard, if not impossible, to understand.
Mireault & Gresham (2015)
Spreadsheet errors are still the rule rather than the exception.
Nixon & O'Hara (2010)
Spreadsheet development must embrace extensive testing in order to be taken seriously as a profession.
Bock (2016)
Spreadsheets are notoriously error-prone.
Cunha, et al (2011)
Spreadsheets have a notoriously high number of faults.
Rust, et al (2006)
Developing an error-free spreadsheet has been a problem since the beginning of end-user computing.
Mireault (2015)
...few incidents of spreadsheet errors are made public and these are usually not revealed by choice.
Kruck & Sheetz (2001)
Spreadsheets are easy to use and very hard to check.
Chen & Chan (2000)
Spreadsheets contain errors at an alarmingly high rate.
Abraham, et al (2005)
Spreadsheets are the most popular live programming environments, but they are also notoriously fault-prone.
Hermans & van der Storm (2015)
It is now widely accepted that errors in spreadsheets are both common and potentially dangerous.
Nixon & O'Hara (2010)
Spreadsheet errors have resulted in huge financial losses.
Abraham & Erwig (2007)
A significant proportion of spreadsheets have severe quality problems.
Ayalew (2007)
Spreadsheets... pose a greater threat to your business than almost anything you can imagine.
Howard (2005)
The software that end users are creating... is riddled with errors.
Burnett & Myers (2014)
Overconfidence is one of the most substantial causes of spreadsheet errors.
Sakal, et al (2015)

Spreadsheet bibliography

Title What we don't know about spreadsheet errors today: The facts, why we don't believe them, and what we need to do
Authors Raymond R. Panko
Year 2015
Type Proceedings
Publication EuSpRIG
Series July
Abstract

Research on spreadsheet errors is substantial, compelling, and unanimous. It has three simple conclusions:

  • The first is that spreadsheet errors are rare on a per-cell basis, but in large programs, at least one incorrect bottom-line value is very likely to be present.
  • The second is that errors are extremely difficult to detect and correct.
  • The third is that spreadsheet developers and corporations are highly overconfident in the accuracy of their spreadsheets.

The disconnect between the first two conclusions and the third appears to be due to the way human cognition works. Most importantly, we are aware of very few of the errors we make. In addition, while we are proudly aware of errors that we fix, we have no idea of how many remain, but like Little Jack Horner we are impressed with our ability to ferret out errors.

This paper reviews human cognition processes and shows first that humans cannot be error free no matter how hard they try, and second that our intuition about errors and how we can reduce them is based on appallingly bad knowledge. This paper argues that we should reject any prescription for reducing errors that has not been rigorously proven safe and effective.

The paper also argues that our biggest need, based on empirical data, is to do massively more testing than we do now. It suggests that the code inspection methodology developed in software development is likely to apply very well to spreadsheet inspection.

Full version Available
Sample
Cell error rates and probabilities of a bottom-line error
Spreadsheet configuration metamodel
The probability of an error increases rapidly when there are many calculations that depend on precedent cells. A spreadsheet with 100 cascade cells and a 3% cell error rate has a 95% probability of containing an error.
Go to top