Excel Hell: It's not just blame for pandemic pandemonium being spread between the sheets

Column The howls of disbelieving, horrified laughter caused by the news of the latest pandemic data cock-up yesterday were well deserved.

16,000 cases lost – purportedly in a blunder involving CSV data, row limits, and an out-of-date Excel file format? In a multibillion-pound, "world-beating" contact-tracing system? Unnoticed for a week of rising infection? In a system known to be broken for months but still not fixed?

Ridicule and despair, those shagged-out nags of our Johnsonian apocalypse, once again trudged exhaustedly across the plaguelands of England.

But the true horror is rooted much deeper and the underlying sins stretch much wider than Number 10. Of that sad catalogue of fail, one item which was widely blamed for the chaos deserves our very finest scorn. One alleged piece of the jagged little jigsaw is a global pandemic all its own, poisoning our data and sickening our businesses for 35 years. In five years' time, with some luck and much work, vaccines and social changes will see COVID-19 demoted to flu status, yet Excel will be with us still.

The government has said that its "mishap" won't "materially alter the course of the pandemic": you and I know that can't be true, not if test and trace is having any effect whatsoever. People will sicken and die because of this mistake, whether it's connected to the cursed spreadsheet or not.

Use spreadsheets for their intended purpose

For your edification, I present this list – 44 pages long – of spreadsheet horror stories, of data entry, calculating, modelling, and analytic chaos that has cost hundreds of millions of pounds and put endeavour of all sorts at risk.

To complete the evidential bundle, I can only refer to my own experiences of a lifetime of dealing with spreadsheet abuse, some of it not even my own, in the sure knowledge that you too will have been a victim. Some research says that more than 90 per cent of spreadsheets have errors in them, and that's before those that think they're databases, or document management systems, or org charts, or security front-ends or… the list is infinite of spreadsheets doing what spreadsheets should not, and doing it extremely badly.

I say spreadsheets, because all are bad, but I mean Excel, because that's the only one that matters. That's the compulsory one. The compatibility enforcer, the one that sets the rules. The one that misrules.

Excel remains recognisably the mutant offspring of VisiCalc, the original 8-bit small business product that started SME computing and made Apple, well, Apple. We are now 40 years on with computers a million times more powerful – 1.5 million times by Moore's Law – yet Excel continues to squat at the heart of business computing like a bowl of leeches in the ICU.

It is an execrable design, from any angle. It has forced generations of untrained business people to operate without help on their data through a letterbox using Lego bricks as scalpels. It knows not any modern data techniques – of structure, robustness, verification, documentation, modularity, versioning, variable typing, variable naming, bound labels. How the Hell is anyone supposed to build and use and communicate and maintain any sort of model where you have to build it cell by cell, with tricksy little links and kiddy algebraic naming conventions?

As for the user interface and accessibility – no condemnation is strong enough. The UI has all the out-of-time impracticalities of Gothic Revival with none of the redeeming aesthetics, the calculating engine can't calculate properly – just look at Wikipedia's list of Excel "Quirks" – and it is wildly version, implementation and platform-dependent. It is a sixth-form programming project grown to the size of Godzilla.

If you mention VBA, I shall scream.

If you mention security, I shall scream louder.

For this, I blame Microsoft. People have to use Excel because it is the only data manipulation tool in Office, and Office is the only game in town. And because of this, Microsoft hasn't done an innovation worth a damn in personal data management for the common business user in decades.

It doesn't have to, and when Microsoft thinks it doesn't have to do something – Internet Explorer 6, m'lud, we have not forgotten nor shall we ever forget – the rest of us can go hang.

The result is an unending history of misery, at a corporate and a personal level, because Microsoft doesn't care. It's got your money, you've got to use its software, off you go. Microsoft can sink billions into vanity projects like quinquennial mobile strategies that nobody asks for, and it loves talking about AI and quantum and all the things.

Geneticists throw hands in the air, change gene naming rules to finally stop Microsoft Excel eating their data

READ MORE

But it hasn't done a decent job – any job – of the real hard problem, the one that needs dedication and innovation and insight and hard, interactive, experimental work: that of creating a business data tool that doesn't actually hurt us.

If there were health and safety rules for software, Excel would be up there with radium cigarettes and arsenic gobstoppers.

In fact, where there are rules – such as in some medical regulatory environments – you can't just use Excel. You have to certify your particular application. But in the absence of regulation, in a distorted monopoly market run by a company still furious it has lost its dictatorial powers elsewhere, nothing will change.

It could be better. Microsoft could atone. It could sponsor design competitions. It could ask the world's finest digital design brains to set up the R&D framework for a revolution. It could look at the damage done and say: "We can do better."

It's a very hard problem. People and data do not mix well, but in the sacred names of Turing, Shannon, and Lovelace, we deserve a better answer than Excel.

We all deserve better than Excel. ®