Extract your data with Daasity

Get an overview of how use Daasity to extract and load data into your data warehouse

Introduction

Daasity makes it easy to connect to all of your data sources, and load your data into a centralized data warehouse and reporting platform.

To start extracting data with Daasity, you just need access to the platforms you want to extract data from, then head straight to the Integrations page in the Daasity app. Each integration has its own specifications (credentials to enter, or files to upload).

Once you've set up an integration, it will be included in your daily data update, or you can configure your custom workflows to manage how often the data is extracted and whether any custom transformation code should run upon completion. You can get all the details on workflows here.

Example of setting up an integration

The video below shows how to add an integration and configure it in a workflow:

Understanding your extracted data

The Integrations section of this knowledge base has the following information for each integration to help you get the most out of your data:

• Integration Setup: Instructions for setting up your integration and troubleshooting setup problems.

• Integration Specifications: Technical details about the integration and the data that each integration will extract and load. These articles include which endpoints Daasity pulls data from, context on all of the tables created by the integration, and an entity relationship diagram (ERD) to help you understand how the different tables are related.

• Workflow Configuration Setup: Context on how frequently an integration can return fresh data. Some integrations will not extract new data more than once per day, while others can get new data hourly.

• Transformation Configuration Setup: Instructions on what transformation code you should add to your script manifest files to transform the raw data from the integration as part of our Daasity Data Model. Read our Transforming Data with Daasity article for more context.

• Important Notes (Not included for all integrations): Important notes and caveats about the data that you should know before diving in to analyze it.

Extraction frequency

When you set up most integrations, they will extract data only once per day by default. The exceptions to this are Shopify and Amazon Seller Central, which are set up to run every hour by default.

If you would like an integration to run an extraction more frequently than once per day, you can do so by setting up a custom workflow, configuring the appropriate refresh interval, and adding the integration to the workflow.

You can configure a custom workflow to run as frequently as once per hour. However, most of our integrations will not get new data every hour. To find out if a certain integration will get new data more frequently than once per day, read the Workflow Configuration Setup doc for that particular integration.

Brand supplied data (BSD)

Our BSD components makes it easy for your business users to provide important information that can be used in data transformations, e.g.: revenue forecasts and SKU attributes.

Business users can provide BSD data either directly through the Daasity app or through a set of standard Google Sheets that we set up for you at the time of account creation. Then Daasity will import that data into the bsd schema in your data warehouse. Learn more about BSDs here.