Most credit union leaders are familiar with the concept of Big Data and business intelligence, but they may fail to fully understand the significance they have on their credit union and its future. Big Data can provide credit unions with the ability to make better decisions that positively affect member relationships and ultimately their top and bottom lines. An essential piece of any business intelligence (BI) strategy is a data warehouse. Data warehouses provide credit unions with the ability to integrate data from many disparate sources to create a single source of truth. From this single source of truth, credit unions are able to generate reporting and analytics tools that leverage data to make the most informed business decisions possible. A data warehouse project seems simple: find all disparate sources of data and consolidate them into a single source of truth. In all actuality, building a data warehouse is a complex process that could end in disaster if handled improperly. There are several obstacles in the process that need to be overcome in order to achieve success. These obstacles typically take an extensive amount of time to conquer, especially the first time they’re encountered. Credit union leaders should consider the following data warehouse challenges before building a data warehouse:
- Data Quality – In a credit union data warehouse, data is coming from many disparate sources from all facets of an organization. When a data warehouse tries to combine inconsistent data from disparate sources, it encounters errors. Inconsistent data, duplicates, logic conflicts, and missing data all result in data quality challenges. Poor data quality results in faulty reporting and analytics necessary for optimal decision making.
- Understanding Analytics – The powerful analytics tools and reports available through integrated data will provide credit union leaders with the ability to make precise decisions that impact the future success of their organizations. When building a data warehouse, analytics and reporting will have to be taken into design considerations. In order to do this, the business user will need to know exactly what analysis will be performed. Envisioning these reports will be difficult for someone that hasn’t yet utilized a BI strategy and is unaware of its capabilities and limitations.
- Quality Assurance – The end user of a data warehouse is using Big Data reporting and analytics to make the best decisions possible. Consequently, the data must be 100 percent accurate or a credit union leader could make ill-advised decisions that are detrimental to the future success of their business. This high reliance on data quality makes testing a high priority issue that will require a lot of resources to ensure the information provided is accurate. The credit union will have to develop all of the steps required to complete a successful Software Testing Life Cycle (STLC), which will be a costly and time intensive process.
- Performance – Building a data warehouse is similar to building a car. A car must be carefully designed from the beginning to meet the purposes for which it is intended. Yet, there are options each buyer must consider to make the vehicle truly meet individual performance needs. A data warehouse must also be carefully designed to meet overall performance requirements. While the final product can be customized to fit the performance needs of the organization, the initial overall design must be carefully thought out to provide a stable foundation from which to start.
- Designing the Data Warehouse – People generally don’t want to “waste” their time defining the requirements necessary for proper data warehouse design. Usually, there is a high level perception of what they want out of a data warehouse. However, they don’t fully understand all the implications of these perceptions and, therefore, have a difficult time adequately defining them. This results in miscommunication between the business users and the technicians building the data warehouse. The typical end result is a data warehouse which does not deliver the results expected by the user. Since the data warehouse is inadequate for the end user, there is a need for fixes and improvements immediately after initial delivery. The unfortunate outcome is greatly increased development fees.
- User Acceptance – People are not keen to changing their daily routine especially if the new process is not intuitive. There are many challenges to overcome to make a data warehouse that is quickly adopted by an organization. Having a comprehensive user training program can ease this hesitation but will require planning and additional resources.
- Cost – A frequent misconception among credit unions is that they can build data warehouse in-house to save money. As the foregoing points emphasize, there are a multitude of hidden problems in building data warehouses. Even if a credit union adds a data warehouse “expert” to their staff, the depth and breadth of skills needed to deliver an effective result is simply not feasible with one or a few experienced professionals leading a team of non-BI trained technicians. The harsh reality is an effective do-it-yourself effort is very costly.