Core Concepts

What exactly does Daasity do?

Understanding Daasity’s platform architecture will help you know where data is coming from and how it’s processed. Daasity is an end-to-end ELT (Extract, Load, Transform) platform, and here’s how the data flows:

1. Extraction (E): Daasity connects to dozens of sources (e-commerce platforms, marketing channels, databases, etc.) using pre-built integrations. Data is extracted via APIs or direct connections on a schedule. For example, Daasity pulls orders from Shopify, ad spend from Facebook, email events from Klaviyo, etc. Extraction jobs retrieve new or updated data since the last sync.

2. Loading (L): The extracted raw data is loaded into your cloud data warehouse. Daasity supports warehouses like Snowflake and Google BigQuery. The raw tables are typically loaded into a schema (often called the “integration” or staging schema) with a structure closely matching the source (for traceability). This means you have a one-to-one raw data copy of each source data set inside your warehouse.

3. Transformation (T): Once raw data is loaded, Daasity’s transformation engine takes over. It runs SQL transformation scripts that cleanse, merge, and reshape the data into the unified schemas and data models. This step normalizes different sources into common tables (like the Unified Schemas for orders, customers, products, etc., and various metrics aggregations). The transformations handle business logic – e.g., merging duplicate records, applying currency conversion, calculating new fields, etc.

4. Data Models & Data Marts: After transformations, your warehouse will have both the unified detailed tables and also any aggregate tables or data marts that Daasity creates for analysis. Unified Schemas are the foundation (detailed, transaction-level data), and on top of them, Daasity can build summarized tables (such as daily sales, customer metrics over time, marketing attribution models, etc.). These are ready to be queried by BI tools or within Daasity’s app.

5. Reverse ETL (Operationalize): Optionally, Daasity can push data back out to other systems. Through features like Audiences (Daasity’s term for reverse ETL), you can take the transformed data (for example, a list of high LTV customers or customers who purchased a certain product) and send it to marketing platforms or other apps. This closes the loop, allowing you to act on insights by feeding cleaned data back into marketing, email, or ad platforms.

Architecture Highlights: The entire process is orchestrated in Daasity’s platform. Daasity handles the scheduling via workflows (see the next section) and monitoring. All of your data resides in your warehouse — Daasity’s role is to keep it updated and structured. The benefit of this architecture is that you have full access to your data (you can write SQL against it, build dashboards, etc.), and Daasity’s predefined schemas accelerate your time to insight by providing a plug-and-play data model.

In summary, Daasity automates the heavy lifting of data pipeline management: it Extracts your source data, Loads it into your warehouse, Transforms it into analysis-ready schemas, and even helps you operationalize that data. The architecture is modular, so you can customize transformations or add new sources as needed without breaking the overall flow. Workflows are the heart of Daasity’s data orchestration.

Platform Architecture & ETL Flow

Workflows allow you to schedule and control when data is extracted and transformed. Here’s what you need to know about Daasity workflows:

• What is a Workflow? It’s essentially a scheduled pipeline. A workflow is a configured sequence that first runs extractions for a set of sources, then runs the transformation scripts once the data is loaded. Instead of manually running dozens of jobs, a workflow automates the entire daily (or hourly) update process.

• Default Daily Workflow: When you first set up Daasity, the system creates a Daily Incremental Workflow for you. This daily workflow is scheduled to run once per day (commonly around midnight in your chosen timezone). It will automatically include all your data sources. Each night, it will pull the latest data from each integration and then update all the transformed tables. This keeps your unified data model and metrics up to date every day without manual intervention.

• Custom Workflows: You can create additional workflows for different purposes. For example, you might have a separate workflow to handle a large historical reload, or one that runs more frequently for a specific data source. Workflows can be scheduled as often as once per hour if needed, or just run on-demand. They are flexible: you choose which integrations and which transform scripts are tied to each workflow.

• Chaining and Dependencies: Daasity allows workflows to be chained. This means you can set one workflow to kick off another when it finishes. For instance, you might separate your workflows by domain (e.g., run an eCommerce data workflow, then a marketing data workflow afterwards). Chaining ensures that, say, your marketing metrics workflow only runs after all the purchase data from the eCommerce workflow is updated.

• Monitoring & Metrics: The Daasity app provides a Workflows dashboard where you can monitor progress and see metrics about each run (like duration, records processed, etc.). If a workflow fails at any step (maybe an API was down or a SQL script error), it will stop and flag an error. This is where Notifications come in to alert you. You can manually re-run workflows or individual steps if needed (for example, rerun just one integration’s extract if it failed).

Core Concepts

Daasity’s platform has some fundamental concepts that are important for understanding your data. Here are the core concepts you should know:

• Unified Schemas: Daasity transforms raw source data into Unified Data Schemas. These are standardized data models (like UOS, UMS, etc.) that combine data from multiple sources into one coherent structure. This means regardless of where the data came from (Shopify, Amazon, Klaviyo, etc.), the essential fields will be in a consistent format and schema. It simplifies analysis by ensuring, for example, every order from any source appears in one Orders table with the same columns.

• Customer Householding: In order to analyze customer behavior across channels, Daasity uses householding logic to link customer records. If the same person exists in multiple systems (with the same email or other identifiers), Daasity can assign them a common Household ID or unified customer ID. This allows you to see metrics like “unique customers” without double-counting the same person across Shopify and Amazon, for instance.

• Daasity Valid Order: Daasity defines a concept of a “valid order” to maintain data quality. For example, test orders, fraudulent orders, or other anomalies can be excluded based on predefined criteria. Many metrics and analyses (like calculating true Order Count or revenue) rely on filtering out non-valid orders to ensure accuracy. Understanding what constitutes a valid order in your dataset will help you reconcile numbers and trust the metrics. Plus, now you can customize the calculation methodology for [Revenue] and [Order] as needed to fit your company’s preferences.

• Timezones and Data Freshness: Daasity allows you to set a data warehouse timezone for scheduling daily workflows. All date/time stamps from sources can be converted or interpreted in a consistent timezone as needed. By default, the platform’s daily workflow runs at midnight in your chosen timezone. Knowing this helps you interpret “today” vs “yesterday” data and how often data is updated.

• Data Security: Your data in Daasity is stored in your own secure data warehouse (BigQuery, Snowflake, Redshift, etc.). Daasity follows best practices to keep data secure and confidential. You maintain control over who can access the warehouse. Additionally, Daasity’s extraction uses read-only access to your sources and does not modify source data.

If you need a deeper dive into any of these (like the specifics of the valid order logic or householding algorithms), you can refer to Daasity’s Help Center articles for more detailed documentation.