Big Data What is the worst part of data engineering? Table of Contents Drivers of change The Challenge of Data Engineering The worst part of data engineering Summary In data engineering teams the list is long and depends on your... Abe Dearmer Read More
Big Data Customer Success Story: Fuze improves query performance by 10x Fuze is a cloud-based communications and collaboration platform provider. Fuze drives workplace transformation for its customers with data from 2.5 million meetings, 20 million calls, and 33 million messages each... Abe Dearmer Read More
Big Data Don’t Use Redshift as an OLTP Database It is common to connect an application framework like Django to Amazon Redshift. This is useful when using Redshift data in your application, i.e. in an OLTP scenario. Since Amazon... Donal Tobin Read More
Big Data Redshift’s Automatic WLM with Query Priority: A First Look at Performance Introduction The AWS team recently released a new feature of their Automatic WLM that we’re really excited about: Query Priorities. This feature aims to address the main limitation of Auto... Abe Dearmer Read More
Big Data Use Routine Vacuuming to Reclaim Unused Space Amazon Redshift databases require periodic maintenance known as vacuuming. Amazon Redshift is based on PostgreSQL, but unlike PostgreSQL, Redshift doesn’t offer autovacuum. So when a row is deleted from a... Abe Dearmer Read More
Big Data Use Column Encoding Adding compression to large, uncompressed columns will have a big impact on cluster performance. Compression accomplishes two things: Reduce storage utilization . Because file compression reduces the size footprint of... Donal Tobin Read More
Big Data Use Amazon Redshift Spectrum for infrequently used data Amazon Redshift launched with disruptive pricing. To compare the cost, we’re looking at the price for storing 1TB of data for one year ($ / TB / Year). With a... Abe Dearmer Read More
Big Data Query Optimization: How to efficiently compare two rows in a SQL query Table of Contents The Simplified Problem A Better Solution A Real-World Example Lessons Learned Query optimization that dramatically reduces runtime for queries which use window functions . The Simplified Problem... Donal Tobin Read More
ETL 26 min Top 14 ETL Tools (Updated Jaunary 2025) Integrate.io lists the 14 best ETL software tools for 2024 based on features, user review scores, and more. Which ETL tool should you choose? Abe Dearmer Read More
ETL 11 min SFTP to Salesforce – Guide to a Secure Integration Integrating data from your systems into Salesforce is a critical process. Here is a guide featuring the SFTP to Salesforce process. Donal Tobin Read More
Data Integration 10 min What Is Operational ETL? Delving into Operational ETL: Learn how it transforms business data handling for enhanced productivity. Donal Tobin Read More