How it all started...
Agile development is certainly not a new notion, and is part of a series of software devleopment models that contrast with the traditional waterfall model. The origins can be traced as far as to 1957, with the idea of “incremental software developments methods” and the collection of lightweight development methods that followed through the 80s and 90s. Concretised into an Agile Manifesto in 2001, the first and fourth of its twelve principles are clear: (a) “satisfy the customer through early and continuous delivery of valuable software” and (b) “business people and developers must work together daily throughout the project”. There was a clear reaction to the heavy waterfall-like development process. Focus on the goal, on the business use, on the capacity to be proactive are well present in this manifesto.
From being a niche philosophy, “Agile” became a real success story in the last 10 years. It lingers on the inherent advantages of the process with respect to human behaviour and ability to forecast, and the enormous flexibility/malleability of well architected software (usage of architectural patterns and modern languages), in contrast to the old monolithic way.
Unfortunately, unless a clear governance is imposed by the management in that direction, most big firms are centrally conditioned by heavy-planned IT developments.
But every alternative has pros and cons. For example, Agile development without a good understanding of how to conduct it in a big organisation, can be too disruptive. On the other hand, plan-driven methodologies were rooted in the need to avoid chaotic software development, guarantee the robustness of new releases and of the evolution of IT inside the firm, and thus provide a kind of confidence in that development. That’s why some characterise the later as “predictive” in opposition to “adaptive” for Agile development. It is a priori reassuring to be able to “predict development”. Now, the World outside is all but predictable and that reassurance might be naïve.
The solution that many (non-IT, but still IT-sensitive for some) managers turned to, aside of the centralised, heavy-planned IT development, was therefore something light, flexible, named a “tool” instead of a system, very visual in its development and capable to do a lot without necessarily programming, namely…spreadsheets, not to say Microsoft Excel™ [1]. Politically speaking, this is a “no-brainer”. No need to start asking for additional resources, justify another system, open a pandora box about the current system, etc… In the same vein, and offering at that time more structure with a database engine and the possibility to be a front-end to Microsoft SQL Server (not anymore), Microsoft Access™ [2], armoured with Visual Basic for Applications (VBA) like Excel, became also a RAD (Rapid Development Application) alternative among others used by proactive managers.
Today, companies accumulate thousands, if not millions, of spreadsheets of very diverse quality. In many cases, they depend on their original author and they are strongly undocumented. "Don't blame the knife for the murder". They are a threat and can be a source of fragility, but mostly because the way they are used...or not used. And there is probably a vicious circle as most IT departments tend to be very strict about the use of network and installed systems, but completely overlook the use of spreadsheets by the company, instead of making sure some best practices and documentation policy also exists for these elements. They are seen as informal tools...although they tend to be used quite officially for reporting, valuation, etc. There are many articles out-there on this risk. Some are provided in the references section below.
What made spreadsheets, i.e. Excel, a major success
Excel is a marvelous tool. Some of us who were there in the 80’s might remember VisiCalc , Framework, Lotus 1-2-3, Borland Quattro, and other types of precursors. Since then, the use of spreadsheets has allowed impressive developments, giving the opportunity to quickly visualise the results of calculations, important when testing a calculation, perform them on large series of numbers and providing an intuitive way to present it, more concretely and more visually for managerial purposes. The key development at the heart of spreadsheets’ appearance was the algorithm developed in 1970 by Pardo and Landau enabling automatic natural order recalculations. Today, spreadsheets are used by every single manager on the planet.
But still, at that time there are some key features that allowed Excel to shine over the rest. Here is a non-exchaustive list.
Goal seek & Solver
Solver was revolutionary. Some of you might remember their courses of Operational Research with concepts such as: Linear programming, Quadratic/Lagrangian optimisation under constraints, Simplex, Northwest Corner algorithm, etc… Well, with the appearance of Excel and Solver, these courses have literally disappeared from business schools (which is not necessarily an excellent idea because you still have to design your models).
WYSIWYG
The capacity to import almost any flat file
And there was VBA... (a precusor to RPA for some)
The discipline that has probably been the most receptive one to these advantages and possibilities was Finance.
The 80s correspond to the first PS2 personal computers of IBM, the first use of mouses, something that Logitech had in the drawer as last idea before closing doors. The 90s correspond to the modern use of Internet, the capacity to download more and more data from the Web, instead of getting Betamax or VHS cassettes from a stock exchange after harrassing them, and starting to read vast amounts of data, and that decade is also the decade of the explosion of the use of derivatives and complex financial instruments, something that deserved to be coded into VBA libraries, for those who were not working in Turbo Pascal/Delphi at that time.
Today, some more recent features are really making Excel an unavoidable application:
(almost) Unlimited rows and columns
Still limited in terms of rows and columns in the 90’s, Excel offers nowadays an almost unlimited capacity to open large datasets.
Table objects
Dynamic formulas
Please connect to the following article for the next chapter: Spreadsheets, love and fear...and solutions.
Notes and References
[1] by Microsoft Corporation. Available from: https://office.microsoft.com/excel.
[2] A quick retrospective is available here: https://en.wikipedia.org/wiki/Microsoft_Access.
[3] It must be noted that Numbers, the alternative proposed by Apple, is displaying blank empty pages where tables can be dropped as objects, by design since its inception. But it lacks the "tools" present in Excel.
[4] Buck, Graham, "Over-reliance on spreadsheets ‘hampers Solvency II compliance’", The Global Treasurer, 2016.
[5] Thomson Reuters, "ARE YOU OVER-RELIANT ON SPREADSHEETS?", 2020.