Table of Contents:
  1. How Slow Dashboards Occur
  2. How to Fix Slow Dashboards
  3. Are You Ready for No Lag Dashboards?

Are you tired of slow dashboards? It’s a problem we hear end-users of BI tools complain about time and time again. Whether you’re an end-user or on the data team that the end-users blame, slow dashboards suck!

With many BI tools now offering their own connectors and lightweight data transformation/preparation layers, slow dashboards are a common pain point across all organizations. The idea of having an all-in-one data suite (connectors, transformation layer, dashboards) sounds appealing, but alas, all that glitters is not gold.

When a company is at the start of its BI journey, using a BI tool's own data connectors can make sense. At this point, the company likely does not have a dedicated BI/data team and they are still only figuring out what insights they want to see from a BI tool. For the most part, unless the BI tool’s connectors are poorly built, slow dashboards are not a problem at this time.

As a company becomes more advanced with its BI, the need for data transformation and preparation becomes more apparent. A common step when starting to do data transformation is to use the BI tool’s transformation offering if they have one. The in-built data integration can and does work for specific use cases. Use cases where only lightweight transformations are required and on small volumes of data. 

“One of our Tableau dashboards was taking over two minutes to load due to a complex business query. By using Integrate.io’s transformation layer prior to data being used in Tableau, the dashboard now loads instantly.”

Jason Gilmore, CTO Dreamfactory

How Slow Dashboards Occur

Slow dashboards are typically caused by the following:

  • Large volumes of data ingestion through a BI tool’s data connectors: BI tools don’t usually store the data they ingest anywhere; this means that they pull the data from the various data connectors every time a dashboard is refreshed.
  • Complex data transformations by a BI tool’s transformation offering: As a BI tool doesn’t store the transformed data anywhere, every time a dashboard is refreshed, the complex transformation must be run before the dashboard can load. 
  • Data transformations on large data sets: The most common example of this is a ‘join’ transformation on a large data set or table. Again, as a BI tool doesn’t store the transformed data, every time a dashboard is refreshed, the ‘join’ transformation must be run before the dashboard can load. 

 

How To Fix Slow Dashboards

Slow dashboards can be fixed by using a BI stack optimized for large data volumes and any complex data transformations. This stack would contain the following:

ETL Tool

  • The ETL tool must be able to handle large volumes of data
  • A rich and robust transformation offering is a must for the ETL tool
  • Bonus points if the ETL tool is a low-code/no-code platform as this will allow non-technical users to build and manage a company’s data pipelines and transformations

Data Warehouse

BI tool

  • With the ETL tool taking care of the data ingestion and transformations, the BI tool is used for what it does best - data visualization.

 

Using this BI stack strategy, the leading causes of slow dashboards are removed: 

  • Large volumes of data being ingested through a BI tool’s data connectors: The ETL tool’s data connectors are built to handle data at scale.
  • Complex data transformations being performed by a BI tool’s transformation offering: With the ETL tool now handling the data transformation and the data being stored in a data warehouse, transformations do not need to be run on the full set of data every time a dashboard is reloaded. Instead, the transformed data is sitting in the data warehouse, waiting to be loaded. 
  • Data transformations on large data sets: While a ‘join’ on a large data set will still take time with an ETL tool, the runtime is typically quicker than on a BI tool, and the transformed data is then stored in the data warehouse waiting to be loaded. 

Are You Ready For No Lag Dashboards?

The key to achieving No Lag Dashboards is finding an ETL tool that can do all the heavy lifting for your data strategy. Integrate.io has helped 100s of customers eliminate their slow dashboards by taking advantage of Integrate.io’s scalability and powerful transformation layer. 

 

What are you waiting for? Schedule a call with our support team and give your team the no-lag dashboards they need.