Business Intelligence
Methodology.
BI Project Management
Best Practices.
Agile
Concepts
There are plenty of talented developers out there, but what makes or breaks the success of a project comes down to BI process flow. It’s common to take the classic waterfall approach, where results aren’t delivered until everything is finished and “perfect.” The problem with this approach is that almost every data project will run into snags. In this day and age, data is complex enough to where almost every organization has a pretty nuanced set of data. And it takes a developer diving into that data to discover those nuances. If data was more straight-forward and clean, we wouldn’t need large EDW or BI teams. But that’s the job of a developer: to discover the nuances and issues in the data and work to resolve them.
So with every data project, you should expect surprises. The way we account for this is with the agile concept of communicating results early and often. We don’t wait six months to communicate that we’ve discovered a nuance in the data that requires a decision. We shoot for weekly or bi-weekly check-ins.
Another challenge to data projects is that it is almost impossible for a non-technical person to write a perfect requirements document. This is because that non-technical person doesn’t work hands-on with the data. They don’t know the nuances of the data, and therefore don’t know what’s possible, or what challenges need to be overcome. This requires a strong collaboration between the non-technical person, who knows what business problem needs to be solved, and the technical person, who has the skills to solve it. This is just another reason why regular communication is key to make sure the project is flowing in the right direction, and that it will provide value when it’s done.
Requirements
Gathering
For a lot of data projects, success or failure can come down to whether the right person was in the room when requirements were gathered. For example, too many EDW projects have low adoption rates because they didn’t consult with the analysts who will be using that EDW. They consult with non-technical staff, who do know what business needs can be solved by an EDW. But those non-technical staff don’t know how the data needs to be structured in order to solve it. That’s where analysts come in.
The key with any data project is to make sure the person who will use the new asset is the main stakeholder for requirements gathering. In the example of the EDW, the analysts and developers who will use the EDW should be the main stakeholders. In the example of new report creation, the business staff who will use that report should be the main stakeholders. This seems intuitive, but it’s often missed. For example, an executive might request that a report be created. But if that person won’t be engaged with the report, then they likely don’t know how the report needs to be arranged.
It’s perfectly fine to include more people in requirements gathering. But it’s absolutely necessary that the main users of the new asset be involved. Or you risk wasting time and money.
Development
If you have the right requirements and the right communication, the rest comes down to logistics. Does our EDW need slow-changing dimension tables? What transformations should we make in our clean environment? These questions are relatively easy to answer when you have the right process in place, and capable developers to get it done.
For new report creation, one key decision point is what platform should be used. SSRS is powerful for regularly scheduled reporting, especially sent by email or dropped in a folder. Power BI is powerful for dynamic engagement with data. It’s much better at allowing you to filter and group data on the fly. We generally don’t suggest you pick just one platform for every report. Instead, choosing the right platform should be part of the requirements gathering.
Report
Conversion
It’s common for organizations to have a catalog of legacy reports in an old platform like Business Objects or Crystal Reports. The easiest way to convert these is to create an SSRS report that functions in the exact same way. But, if you want to tackle the larger project, you could consider these tasks as well:
-
Report cataloging
-
Converting data sources from legacy to EDW
-
Consolidating multiple reports into one
-
Considering Power BI for more dynamic options
All of these options should be planned carefully. It’s much easier to estimate a set amount of time for each report to be converted to SSRS. It’s much harder to calculate a reliable estimate for converting to an EDW. This requires data analysis to understand the differences between the legacy source and the EDW. It involves new query writing and regression testing. This is how a report conversion project can balloon from a 1 month estimate to a 6 month estimate.
