The statistical data concerning implementation of projects of different size, using agile methodologies, indicate on the average 350% more successful implementations in comparison with the cascade approach. In the large project category, as many as 600% more Agile implementations were successful in comparison with standard cascade implementations (source: CHAOS Report 2015).

However, what is actually the Agile approach that everybody talks about, but hardly anyone applies in our BI area? ‘Quicker, better, cheaper’. A beautiful slogan which had been yelled out by the leading supporters of this method over the last decade. This style of working has already been adopted mainly in organisation of programming in transactional systems, or areas where iterative product development and service improvement cycles are something natural. In these areas, implementation of the methodology only required adaptation for the purpose of transferring processes into the digital space. Our BI (Business Intelligence) is still a no man’s land, where few pioneers are trying to ‘domesticate’ this methodology. Fortunately, or unfortunately, following the numerous disasters of projects executed by means of the standard approach, as well as other factors facilitating their implementation using the cascade model, the corporation departments are becoming increasingly positive about the agile methodologies.

Let us wonder for a while about why the popularity of agile approach in BI does not match the rate at which it gains popularity in transactional systems. For the solutions where one of the purposes is data presentation, implementation of this functionality is limited to simple column preparation in the database, and then mapping this column/entry to the pre-prepared field in the user’s front end. Puff! And there it is. Simple. Quick. Effective. And ready for quick change, if the Customer wanted a different option for the following iteration.

However, this works differently in our field. Supply of a new attribute in the Data Warehouse is usually connected with preparation of several processes (for extracting, cleaning, integrating and loading) and dimensioning of a set of data containing a particular element, which is to be finally placed in the simple quick above-mentioned column. The entire logics is underneath, and it is precisely us to shape and deliver it, because as a principle, it constitutes a basis for any subsequent operations in the system operating on data. In order to recognise a particular data area as complete, between several and even up to several dozen jobs/processings must be usually written. There are many logical and architectural layers for quick management and development by a team that wants to meet the Customer’s expectations. And this is precisely the fundamental difficulty in adapting agile methodologies – a complex network of connections that must be untangled using concepts in a systematic manner, in order to finally combine it into a functioning whole.

On the other hand, due to the advantages visible at first sight (which I will later write about), there are teams that want to experiment with the agile approach.  The teams that modify the generic agile approach frequently create hybrids which are slightly more complex, but NOT LESS EFFECTIVE! In practice, proper application of agile methodology in integration and data visualisation projects caused a decrease of defects to ZERO, while maintaining a stable and frequent increase of business value of the manufactured product. Not to mention the reduction of budget and labour intensity expenditure incurred within the standard obsolete cascade approach (i.e. something that had been manufactured in 1896).

For those who are only beginning their adventure with agility, here is some information on how key issues related to cascade approach are addressed by the agile approach.

1. What are the problems of traditional (cascade/waterfall) approach?
2. Poor quality of delivered solution.
3. Barely visible goal of the project.
4. Too large project risk.
5. Stiffened and problem causing approach to handling changes.

How are these problems faced by the agile approach?

1. The quality improves from the first day of work (… because the tests already start on the first day of work).
2. The entire project is visualised already during the first sprint. The solution is materialised at the end of each iteration/sprint.
3. The risk is limited, because potential issues are quickly detected (the so-called Fail-Fast – the quicker we fall, the quicker we get up).
4. The customer is satisfied, because he is free to implement changes during the project duration without incurring any additional, unpredicted costs.

The question that should be asked now is not ‘is it worth it?’, but ‘when do we enter the agile path?’

With this article, I am beginning a cycle dedicated to implementation of agile methodologies in the BI area.