Agile BI and Analytics
Frustration on the part of business users often leads to everyone having his or her own private data store in the form of Excel spreadsheets or Access databases.
In one company, the COO had 4 years of month-end data in one spreadsheet file on her local hard drive. In another large, multi-billion dollar company the CFO presented monthly financial reports to his leadership team consisting of bar and line graphs on powerpoint slides that he manipulated to represent data to fit his narrative of results
Adding to the pressure on reporting organizations lately is the plethora of SaaS data visualization and reporting/analytics tools available to end users. Now, not only do people have their own local data stores, they are able to present information with more sophisticated visualizations that infer validity and analytical rigor.
The answer for IT organizations trying to establish data governance and security while meeting the BI and analytics needs of the organization is a more Agile/Lean approach.
Traditional reporting process has all the attributes of a failed waterfall software development process: up-front design; heavy reliance on technical documentation; and separation of the end user from the solution provider. Outcomes typically reflect that process: missed expectation; disappointed and frustrated end users and solution providers; rework; and long lead times.
An Agile/Lean approach to BI applies principles and practices that have proven effective in software development, including:
- frequent face-to-face interaction between end users and solution developers
- regular delivery of finished work in short cycles
- prioritizaiton of work according to business value
- within and between delivery cycles, review of the work backlog to re-assess business value and prioritize requests
- visibility to all stakeholders of the work backlog, work in progress, and latest completed work
- Reduced time to deliver. In one example I saw an accounting team reduce the number of hours required to deliver their month-end account statement from 40 to 4, a 90% reduction in time to complete.
- Improved accuracy. Any report that relies on manual data manipulation (copy, paste, formulas embedded in cells, etc.) is bound to introduce inadvertent errors. In the month-end statement of accounts example, errors had been introduced to cells in the spreadsheet and had gone undetected for over a year. The resulting erroneous results had been perpetuated that entire time. Correcting that data and putting security and controls around it radically improved data accuracy and ensured accuracy from that point forward.