Setting up managed data pipelines with Fivetran saves a lot of development work. Building an average custom connector to a source system takes roughly two hundred hours of development. Once a connector has been developed, a connector must also be managed. Suppose that changes take place in the diagram of the source system, it is possible that the loading procedure no longer functions properly. All in all, it can be an enormous time investment to develop and manage custom data pipelines.
Fivetran, on the other hand, can be rolled out within five minutes. All you have to do is set up a connection to your data warehouse and then with a few clicks you can add connectors to a wide variety of source systems. In this example we are going to load Google Analytics data to Snowflake with Fivetran. If you have a Snowflake data warehouse, Fivetran is directly available from Snowflake Partner Connect and vice versa you can also set up a new cloud data warehouse directly from Fivetran.
Loading data in Snowflake with Fivetran
In a few simple steps we will show how easy it is to start using Fivetran’s managed data pipelines from Snowflake Partner Connect.
Once logged in to Snowflake you can start a 14-day Fivetran trial for free via Snowflake Partner Connect. Pay attention; this is only possible if you have the ACCOUNTADMIN role at your disposal.
In Snowflake Partner Connect you have the choice between multiple tools to load data as well as tools to visualize data. From Partner Connect you can directly create a trial account with a few clicks with a direct connection to your Snowflake account. As a result, you do not have to waste time configuring and testing the connection and all permissions are immediately set to be able to use the database.
If you click on Fivetran you will see the following screen. Here you confirm that your name and email address are taken over by Fivetran to create a trial account. To make a direct connection with Snowflake, Fivetran creates a Database, a Warehouse, a User and a Role in Snowflake. The objects created by Partner Connect can be recognized below by the PC prefix.
You will then be redirected to the Fivetran website, where we can complete the activation of the trial and start loading data.
Fivetran is directly connected to the data warehouse, so now we can start adding a connector. We click on “Create your first connector” to continue to the connector overview.
Here we get an overview of all available connectors in Fivetran, the connectors are subdivided into Applications, Databases, Events, Files and Functions.
In the search bar we type Google, and then click on Google Analytics.
The first thing we need to do to set up a connector is to choose the scheme to which the data is to be written in Snowflake and a table in that scheme. After this we can authorize the connection to Google Analytics via the Google authorize API.
This gives Fivetran access to viewing your Google Analytics data. This means that access is granted to all properties in the Google account.
If the authentication is successful, we can then choose whether all accounts should be synchronized, or whether we want to select a single and / or multiple properties to synchronize. We can now indicate which measurements and dimensions must be retrieved. After this we save the connector with the push of a button and we test whether the data can be downloaded. If this succeeds, the initial sync starts.
In the last screen we now get an overview of the synchronization that is taking place and we can see when synchronizations have taken place. If the synchronization has been successful, we will receive a confirmation by email.
With a simple select statement we can now see that the data is loaded in Snowflake. Fivetran adds its own Fivetran ID with which the data can now be incrementally loaded.
We hope that you, like us, have become incredibly enthusiastic about Snowflake and Fivetran. If you want to know more about Snowflake and Fivetran, or how you can then easily visualize the data? Then don’t hesitate to contact us