How to Migrate Salesforce Data from Heroku

When using Heroku, it is common practice to transfer or back up data to external locations. While there are many steps in this process, creating an efficient pipeline is relatively simple once you understand all the moving parts. In this article, we show you how to migrate data from Heroku to Amazon S3. In this particular case, we will be connecting and transferring Salesforce data.

Introduction to Salesforce

Salesforce is a well-known customer relationship management (CRM) software that uses cloud technology to offer organizations big data automation and customer management solutions. Salesforce has an innovative team robust capabilities and easily integrates with hundreds of different software and tools.

Let's take a quick look at what makes a SaaS like Salesforce different from the typical CRM and one of the most popular software companies globally.

Salesforce.com was one of the earliest innovators to take localized CRMs and move them to the cloud. This meant you could access your cloud computing and metadata anywhere in the world, on any device with an internet connection in real time. It also meant you wouldn't have to install anything on premise for your company's information technology team.

Salesforce is also a platform that can connect to almost any other cloud-based tool or software, leading to its position as one of the most interoperable tools in existence. For this reason, Salesforce customers have various use cases with their primary CRM tool, from marketing and visualization automation suites like Marketo, Hubspot, and Pardot, to inventory management systems such as Netsuite and Epicor.

Companies often want to take their data stored inside Salesforce and move it to an external data warehouse. Below, we explore how to move Salesforce data to AWS S3 while using Heroku.

Introduction to Heroku

Heroku is a PaaS (platform as a service) that allows you to build and deploy web applications quickly. Heroku supports several programming languages, including Java, Node.js, Python, Ruby, and PHP. Heroku is similar to AWS Elastic Beanstalk in how it abstracts the underlying infrastructure of how you deploy your code.

Unlike other PaaS providers like AWS, Heroku is also unique because it uses a Git-based workflow for pushing and managing changes on its platform. This allows developers during application development to compute or create commits easily and branch off their work if they need multiple ongoing updates without having too much trouble keeping track of all this activity within one master branch.

Transferring Salesforce Data From Heroku to Amazon S3

To transfer Salesforce data to Amazon S3 while using Heroku, you must first set up the connection with Salesforce and Heroku.

The easiest way to achieve this is through Heroku Connect.

Heroku Connect is a data integration tool that allows you to transfer your Salesforce Datasets into Heroku. It does this by taking the changesets in your database and moving them to another destination like Amazon S3, Big Query, or even Parse Server.

Heroku Connect can be used as a standalone product, and it also integrates with other data sources such as the mySQL database Heroku Postgres. You can automatically trigger updates via streaming replication between two different databases. This gives users powerful automation capabilities to manage their data pipelines.

How to Connect Salesforce and Heroku

Setting up your Salesforce Connection through Heroku Connect is relatively easy if you understand how Salesforce works.

Here is a step-by-step process.

Step 1: Create a New Heroku Application

The first thing you want to do is set up a new Heroku application. Do this by logging into Heroku and clicking on the + sign in the top-right corner. Afterward, select the option for "Create App."

Enter a name and then a region for this app.

Step 2: Add Heroku Connect

After you create your app, you will automatically be sent back to your Heroku dashboard. Next, go to the "Resources" tab on the top menu.

You will now have the option to search for any add-on you want to use. In this case, you will use Heroku Connect.

Type in "Heroku Connect" and select "Provision" to add Heroku Connect to your application.

Step 3: Add Heroku Postgres to mySQL Database

After adding Heroku Connect, you will need to add one more add-on called "Heroku Postgres."

Repeat the process taken in step 2, but use the "Hobby Free Dev Plan in the provisioning."

Step 4: Connect Your Heroku and Salesforce Accounts

Now that you have successfully installed the add-ons, you are ready to connect your Salesforce and Heroku accounts.

Click on the "Heroku Connect Resource," and you will be redirected to a new page. To establish your Heroku Salesforce link, click on "Set Up Connection."

Follow the process by setting up a schema and clicking on "Next" to secure the integration.

Next, select "Environment" and choose "Production." Select your API and then authorize the connection.

After this is complete, you will be asked to log back into your Salesforce account. Enter your username and password, and then your Heroku and Salesforce integration will begin.

How to Migrate Data From Heroku to Amazon S3

Now that your Heroku and Salesforce accounts are connected, it is good practice to frequently back up information and data from Heroku to Amazon S3. Transferring data between the two is common.

A great app to streamline this process is called Buckateer.

Here is a step-by-step guide to using Buckateer to integrate Heroku and AWS S3.

Step 1: Download Buckateer to Your Heroku Apps

Follow the same steps described above to provision the Buckateer app.

Step 2: Set Up the Local Environment

After the Buckateer app has been successfully provisioned, it is time to set up the local environment. After you've deployed the service, use locally synchronized configuration vars to duplicate the config vars in your development environment so that it can operate against the service.

Use the Heroku Local command-line tool to set up, execute, and monitor process types defined in your app's Profile.

The Heroku local environment is configured via a .env file. To see all of your app's config vars, type "Heroku config." 

Step 3: Use AWS CLI

The next step in the process is using AWS CLI.

AWS CLI is a unified tool to manage your AWS services.

Once you've installed it, you may modify the Bucketeer settings in your app config or through your SSO page. Here is a full breakdown of how you can properly configure your SSO page.

Step 4: Upload Files

Use the AWS S3 cp command with the bucket URL to upload files. Anything uploaded utilizing the public/ prefix is immediately available. Other files will need to be set to a public access control list (ACL).

Use Integrate.io to Clean Your Data Before Transfer

Before you transfer your Salesforce data from Heroku to Amazon S3, you will want to ensure your data is transformed and cleaned. Using an ETL such as Integrate.io is one of the easiest ways to do this.

Integrate.io uses a combination of advanced algorithms and machine learning technology to analyze the information in your Salesforce database before it gets pushed into Amazon S3. This means that there is no need for manual intervention or heavy maintenance every week, month, etc.

Here is a step-by-step guide on how you can integrate Integrate.io and AWS S3.

How Integrate.io Can Help

Connecting Heroku and Amazon S3 is a beneficial strategy to help you save time and money, especially if data is erased or damaged on your Heroku platform.

Backing up information is an essential part of dealing with large amounts of data, but with that come specific challenges that you can easily alleviate by using a tool such as Integrate.io.

Ensuring that the data aligns between Salesforce and Amazon S3 can be a cumbersome task, but with Integrate.io's ETL capabilities, you can clean your Salesforce data before transferring it into Amazon S3. This cleaning ensures that the transferred information is accurate and up-to-date without worrying about manual intervention or maintenance afterward.

If you are interested in how Integrate.io can streamline your data pipeline, sign up for a free 7-day trial today.