Daasity Knowledge Base
Daasity Knowledge Base
  • Start here
    • Daasity Overview
    • Important Concepts
      • Attribution Overview
      • What is a Daasity Valid Order?
      • Customer Householding
      • Timezones in Daasity
      • Ecommerce Glossary
      • Daasity Data Security Overview
    • Retail Analytics 101
      • Understanding the Basics: POS Data 101
      • How to use Syndicated Data
    • Product Changelog
    • Feature Requests
  • 🔌Extract
    • Extract your data with Daasity
    • All Integrations
      • Alchemer
        • Integration Setup
        • Integration Specifications
        • Workflow Configuration Setup
        • Transformation Configuration Setup
      • Algolia
        • Integration Setup
        • Integration Specifications
        • Workflow Configuration Setup
        • Transformation Configuration Setup
      • Amazon Advertising
        • Integration Setup
        • Integration Specifications
        • Workflow Configuration Setup
        • Transformation Configuration Setup
        • Where to Find Your Data
      • Amazon Seller Central
        • Validating Your Amazon Data in Daasity
          • Comparing Your Amazon Seller Central Orders to Daasity
        • Integration Setup
          • Configure Amazon Reports
          • Monitor Amazon Reports
        • Integration Specifications
          • Orders API
          • FBA Customer Concessions Reports
          • FBA Sales Reports
          • FBA Inventory Reports
          • FBA Removals Reports
          • Inventory Reports
          • Order Tracking Reports
          • Performance Reports
          • Returns Reports
          • Settlement Reports
          • Seller Retail Analytics Reports
        • Workflow Configuration Setup
          • Orders API
          • FBA Customer Concession Reports
          • FBA Inventory Reports
          • Performance Reports
          • FBA Sales Reports
          • FBA Removal Reports
          • Inventory Reports
          • Order Tracking Reports
          • Returns Reports
          • Seller Retail Analytics Reports
          • Settlement Reports
        • Transformation Configuration Setup
          • Orders API
          • FBA Customer Concessions Reports
          • FBA Inventory Reports
          • Performance Reports
          • FBA Sales Reports
          • FBA Removals Reports
          • Inventory Reports
          • Order Tracking Reports
          • Returns Reports
          • Seller Retail Analytics Reports
          • Settlement Reports
      • Amazon Vendor Central
        • Integration Setup
        • Integration Specifications
        • How to use
        • ASIN Mapping Configuration Setup
        • Workflow Configuration Setup
        • Transformation Configuration Setup
          • Vendor Sales & Inventory Report
          • Vendor Traffic Report
      • AppLovin
        • Integration Setup
      • Attentive Mobile
        • Integration Setup
        • Integration Specifications
        • Workflow Configuration Setup
        • Transformation Configuration Setup
      • Azure SQL Database
      • Back In Stock
        • Integration Setup
        • Integration Specifications
        • Workflow Configuration Setup
        • Transformation Configuration Setup
      • Big Commerce
        • Integration Setup
          • Connect Daasity From BigCommerce Marketplace
        • Integration Specifications
        • Workflow Configuration Setup
        • Transformation Configuration Setup
      • BigQuery Datasource
      • Bing Ads
        • Integration Setup
        • Integration Specifications
      • Circana
        • Integration Setup
        • Integration Specifications
        • Data Release Calendar
        • H-E-B Circana
      • Cordial
        • Integration Setup
        • Integration Specifications
        • Workflow Configuration Setup
        • Transformation Configuration Setup
      • Criteo
        • Integration Setup
        • Integration Specifications
        • Workflow Configuration Setup
        • Transformation Configuration Setup
      • D​ata File (CSV, XLS)
      • Emplifi
        • Integration Specifications
      • Extensiv (Skubana)
        • Integration Setup
        • Integration Specifications
        • Workflow Configuration Setup
        • Transformation Configuration Setup
      • Facebook Ads
        • Integration Setup
        • Integration Specifications
        • Workflow Configuration Setup
        • Transformation Configuration Setup
        • Where to Find Your Data
      • Fairing V2
        • Integration Setup
        • Integration Specifications
        • Workflow Configuration Setup
        • Important Notes
      • Google Ads
        • Integration Setup
        • Integration Specifications
        • Workflow Configuration Setup
        • Transformation Configuration Setup
        • Important Notes
      • Google Analytics 4
        • API vs BigQuery
        • API Integration
          • Integration Setup
          • Integration Specifications
          • Workflow Configuration Setup
          • Transformation Configuration Instructions
          • Custom Reports
          • Validating GA4 Data in Daasity
          • Blending Universal Analytics & GA4 Data
        • BigQuery integration
          • Integration Setup
          • Integration Specifications
          • Workflow Configuration Setup
          • Transformation Configuration Instructions
          • Important Notes
          • Creating a Custom Report
      • Google Sheet
      • Gorgias
        • Integration Setup
        • Integration Specifications
        • Workflow Configuration Setup
        • Transformation Configuration Setup
        • Important Notes
      • Impact Radius (Beta)
        • Integration Specifications
      • Iterable
        • Integration Setup
        • Integration Specifications
        • Workflow Configuration Setup
        • Transformation Configuration Setup
      • Klaviyo Summary Statistics
        • Integration Setup
        • Integration Specifications
      • Klaviyo V2
        • Integration Setup
        • Integration Specifications
      • KnoCommerce
        • Integration Setup
        • Integration Specifications
      • Loop Returns
        • Integration Setup
        • Integration Specifications
        • Workflow Configuration Setup
        • Transformation Configuration Setup
      • Microsoft SQL Server Database
      • Mongo Database
      • MySQL Database
      • Narvar
        • Integration Setup
        • Integration Specifications
        • Workflow Configuration Setup
        • Transformation Configuration Setup
      • NetSuite
      • Newstore
        • Integration Setup
        • Integration Specifications
        • Workflow Configuration Setup
        • Transformation Configuration Setup
      • Northbeam
        • Integration Setup
      • Okendo
        • Integration Setup
        • Integration Specifications
        • Workflow Configuration Setup
        • Transformation Configuration Setup
      • Ometria
        • Integration Setup
        • Integration Specifications
        • Workflow Configuration Setup
        • Transformation Configuration Setup
      • Order Desk
        • Integration Setup
        • Integration Specifications
        • Workflow Configuration Setup
        • Transformation Configuration Setup
      • Pepperjam
        • Integration Setup
        • Integration Specifications
        • Workflow Configuration Setup
        • Transformation Configuration Setup
      • Pinterest
        • Integration Setup
        • Integration Specifications
        • Workflow Configuration Setup
        • Transformation Configuration Setup
      • Postgres Database
      • Postscript (beta)
        • Integration Setup
        • Integration Specifications
        • Workflow Configuration Setup
        • Transformation Configuration Setup
        • Important Notes
      • Recharge
        • Integration Setup
        • Integration Specifications
        • Integration Specifications (2021-11 Beta)
        • Workflow Configuration Setup
        • Transformation Configuration Setup
        • Recharge Data and Known Discrepancies
      • Retail Next
        • Integration Setup
        • Integration Specifications
        • Workflow Configuration Setup
        • Transformation Configuration Setup
      • Rockerbox
        • Integration Setup
        • Integration Specifications
        • Workflow Configuration Setup
        • Transformation Configuration Setup
      • Salesforce Commerce Cloud
        • Integration Specifications
        • Workflow Configuration Setup
        • Transformation Configuration Setup
      • SFTP
      • ShipBob
        • Integration Setup
        • Integration Specifications
      • Shipstation
        • Integration Setup
        • Integration Specifications
        • Workflow Configuration Setup
        • Transformation Configuration Setup
      • Shopify
        • Integration Setup
        • Integration Specifications
        • Workflow Configuration Setup
        • Transformation Configuration Setup
        • Shopify B2B Data
      • SKIO
        • Integration Setup
        • Integration Specifications
      • Snapchat
        • Integration Setup
        • Integration Specifications
        • Workflow Configuration Setup
        • Transformation Configuration Setup
      • SPINS
        • Integration Setup
        • Integration Specifications
        • How to Use
        • Data Release Calendar
      • Stay Ai
        • Integration Setup
        • Integration Specifications
      • Target Plus
        • Integration Setup
      • Thankful
        • Integration Setup
        • Integration Specifications
        • Workflow Configuration Setup
        • Transformation Configuration Setup
      • TikTok
        • Integration Setup
        • Workflow Configuration Setup
        • Transformation Configuration Setup
      • Walmart Marketplace
        • Integration Setup
        • Integration Specifications
      • Whole Foods Market
        • Integration Setup
        • Integration Specifications
        • How to use
      • Yotpo
        • Integration Setup
        • Integration Specifications
        • Workflow Configuration Setup
        • Transformation Configuration Setup
      • Zendesk
        • Integration Specifications
        • Workflow Configuration Setup
        • Transformation Configuration Setup
    • Brand Supplied Data (BSD)
      • BSD Overview
      • Monitoring Your BSD Progress
      • Revenue Plan
      • Marketing Budget
      • Marketing Spend
      • RFM Values
      • Shipping & Fulfillment Cost
      • SKU Hierarchy
      • SKU Mapping
      • SKU Cost
      • Discount Code Mapping
      • Other Order Sources
      • Channel Mapping
        • Overview & Setup
        • Filling out Channel Mapping
        • Missing from GA & Missing from BSD
      • Configuration
  • 📊VISUALIZE
    • Dashboards
      • Dashboard Overview
      • Standard Dashboards
        • Daasity 101 Overview Dashboards
          • Flashes
          • Shopify Sales Report
          • Account Health
        • Digital Product & Revenue
          • Orders and Revenue
          • Amazon Settlement Report
          • Product Repurchase Rate
          • Product
            • Product Dimensions
              • Why is a Product Name NULL?
          • Inventory
          • Operations
            • Fulfilment Status Options
        • Retention Marketing Dashboards
          • Notifications (Email/SMS)
          • Retention
          • Customer Performance (Email/SMS)
          • Site Funnels
        • Acquisition Marketing Dashboards
          • Attribution Deep Dive
            • Marketing Attribution
              • Attribution Overview
              • Survey-Based Attribution
              • Discount Code Attribution
              • First Click, Last Click, and Assisted Attribution
              • Custom Attribution
              • Vendor-Reported Attribution
              • CPO, CPA, and CAC
              • Missing from GA & Missing from BSD
          • LTV & RFM
            • RFM
              • RFM in the Daasity App
              • Understanding RFM
              • Time on Books (TOB)
          • Marketing
          • Vendor-Reported Marketing Performance
          • Site Analytics & Attribution
      • Integration Dashboards
        • Amazon All Orders Report
        • Amazon Seller Central Business Reports
        • Gorgias
        • Klaviyo Campaign & Flow Performance
        • Loop Returns
        • Okendo Reviews
        • ShipBob
        • Shipstation
        • Subscription
        • Walmart Marketplace Account Sales Report
        • Yotpo
    • Explores
      • Explore Overview
      • Order Line Revenue vs. Transactional Sales Report
      • How to Access An Explore
      • Standard Daasity Explores
        • Current Inventory Level Explore
        • Customer Details Explore
        • Daily Company Metrics Explore
        • Daily Marketing Plan To Actual Marketing Performance Explore
        • Daily Plan to Actual Explore
        • Lifetime Value Explore
        • Marketing Attribution Explore
        • Multi-Channel Transaction Explore
        • Order and Order Line Revenue Explore
        • Product Page Explore
        • Shopping Stage Explore
        • SKU Level Repurchase Rates Explore
        • Traffic Explore
        • Transactional Sales Report Explore
        • UA vs G4 Explore
      • Integration Explores
        • Algolia
        • Amazon Settlement Explore
        • Fairing Survey Responses Explore
        • Gorgias Tickets Explore
        • Loop Returns Explore
        • Okendo Reviews Explore
        • Subscribers and Subscription Monthly Churn Rates Explores (Recharge)
        • Shipstation Explore
        • Yotpo
        • Zen Desk
    • Looker Guides
      • Building visualizations
      • Working with Date Dimensions
      • Schedule a Report or Dashboard
      • Exporting Data
    • Looker Guides (Developer)
      • Daasity Hub & Spoke Introduction
      • Hub & Spoke File Structure
      • Working with LookML Dashboards
      • Customizing Base Daasity Code
    • Visualize using other BI tools
  • 🔀Transform
    • Transforming Data with Daasity
    • Daasity Data Models
      • Daasity Data Model Overview
      • Unified Data Schemas
        • Unified Schema Introduction
        • Unified Order Schema (UOS)
        • Unified Notification Schema (UNS)
        • Unified Subscription Schema (USS)
        • Unified Marketing Schema (UMS)
        • Unified Traffic Schema (UTS)
        • Unified Wholesale Schema (UWS)
      • Data Mart Models
        • Data Mart Introduction
        • Data Reporting Platform (DRP)
        • Channel Attribution Data Mart
        • Marketing Data Mart
      • Transform Code
        • Transform Code Introduction
        • Transform Code Sequence - DRP
        • Initialization Code
        • Householding Code
        • Order Status Code
        • Order Channel and Marketing Code
        • Daily Plan Code
        • Order and Order Line Revenue
        • Plan to Actual Code
        • Customer Code
        • Subscription Code
        • Reviews
        • Life-Time Value Code
        • Returns
        • Product and Inventory Code
        • Email & SMS
        • Transform Code Sequence - Data Mart
        • Unified Order Schema (UOS) Code
        • Unified Marketing Schema (UMS) Code
        • Unified Notification Schema (UNS)
        • Channel Attribution Data Mart Code
        • Marketing Data Mart Code
      • Source to Final Destination
    • Code Repository
      • Code Repository Overview
      • Getting Access to Your Code Repository
      • Code Repository Branches
      • Custom SQL Code
      • Custom Script Manifest Files
    • Script Manifest (YAML) Files
  • 🎵Data Orchestration
    • Workflows
      • Getting Started with Workflows
      • Creating Workflows
      • Managing Workflows
      • Workflow Metrics
    • Notifications
      • Introduction to Notifications
      • Enabling Notification Emails
      • Notifications UI
      • Managing Notifications
      • Authorization List
  • Audiences
    • Audiences Overview
    • Pushing Audience Data to Targets
      • Targets Overview
      • Attentive
        • Events Target
        • Subscribers Target
        • Target Endpoint Documentation
      • Facebook Ads
        • Facebook Target
        • Target Endpoint Documentation
      • Google Ads
        • Google Ads Target
        • Target Endpoint Documentation
      • Klaviyo
        • Events Target
        • Profiles Target
        • Target Endpoint Documentation
      • Sailthru
        • Events Target
        • Users Target
        • Target Endpoint Documentation
      • Tiktok Ads
        • TikTok Target
        • Target Endpoint Documentation
    • Building Audiences
  • warehouse
    • BigQuery
    • Redshift
    • Snowflake
    • Test Data Warehouse
      • Test Data Warehouse Overview
      • Create a Snowflake Test Warehouse
      • Create a Redshift Test Warehouse
  • —
    • Go to Daasity App
    • Contact Support
    • ✍️Feature Requests
Powered by GitBook
On this page
  • What are script manifest files?
  • The daily_incremental.yml template file
  • Other template manifest files
  • Working with script manifest files
  • Structure
  • YAML formatting
  • Referencing script files
  • Referencing integrations
  • Manifest files must be in root folder

Was this helpful?

  1. Transform

Script Manifest (YAML) Files

This page provides an outline of how to setup a Script Manifest File within the Daasity App to configure the transformation that will execute as part of a workflow

PreviousCustom Script Manifest FilesNextWorkflows

Last updated 8 months ago

Was this helpful?

What are script manifest files?

Script manifests are files used in to dictate which transformation scripts should run when. They are YAML files that list a sequence of SQL scripts that should be executed in a particular order.

Script manifests can be used to run SQL scripts located in your custom code repository or in one the Daasity shared script repositories.

The daily_incremental.yml template file

When you get set up with Daasity, you are automatically set up with a that is tied to a daily_incremental.yml script manifest file. This file by default is blank (except for a set of comments).

If you want to get a jumpstart on setting up your daily incremental transformation code, you can copy and paste some or all of the code from (located in our shared code repository) into your custom code repo's daily_incremental.yml file.

Other template manifest files

This folder currently has the following sample script manifest files:

Working with script manifest files

Structure

Here is a basic example of a daily_incremental.yml. This example file would run a set of scripts if you have a Shopify integration and another set of scripts if you have a Google Analytics integration:

daily_incremental.yml
version: "2.0.0"

sections:
  shopify_stuff:
    conditional_integrations: "shopify" # Makes it so the scripts in this section will only run if you have an active Shopify integration
    scripts: 
      - "some_shopify_script_in_your_repo.sql"
      - "github://platform-sql-shared/scripts/some_shopify_script_in_the_Daasity_shared_repo.sql"
      
  google_analytics_stuff:
    conditional_integrations: "google_analytics" # Makes it so the scripts in this section will only run if you have an active Google Analytics integration
    scripts: 
      - "some_google_analytics_script_in_your_repo.sql"
      - "github://platform-sql-internal/scripts/some_google_analytics_script_in_the_Daasity_internal_repo.sql"
      
    

This example has the following important components to every script manifest file:

  • version - The current version number. This should not be changed from "2.0.0".

  • sections - This key must be included, and should only be included once. Each section within it is just a way to group a set of scripts together.

        • * There is a legacy integrations tag that works similarly and is still supported. The legacy tag supports a list of integrations, all of which are treated with OR logic, i.e. if any of the integration in the list is active and authorized in the account, the scripts in the section will run. You should not combine an integrations and a conditional_integrations tag in the same section.

YAML formatting

It is important to pay attention to your file formatting or your script manifest file may not work.

  • Indentation in the file matters!

  • Each level of indentation needs to be 2 spaces (not tabs)

    • version: & sections: should not be preceded by any spaces

    • The section name should be indented 2 spaces from the beginning of the line

    • integrations: and scripts: should be indented 4 spaces from the beginning of the line

    • Individual script and integration list items should be indented 6 spaces from the beginning of the line

  • Comments may be specified using a hash symbol #

Referencing script files

In the example file snippet above, you can see that there are two different types of file paths for the SQL scripts being referenced.

Local script files

- "some_shopify_script_in_your_repo.sql"
- "some_google_analytics_script_in_your_repo.sql"

This is how you would reference files that exist in your merchant repo. In this example, both files are in the top-level folder of your merchant repo. If you are going to be referencing custom scripts, they will look like these examples.

Remote script files

- "github://platform-sql-shared/scripts/some_shopify_script_in_the_Daasity_shared_repo.sql"
- "github://platform-sql-internal/scripts/some_google_analytics_script_in_the_Daasity_internal_repo.sql"

This is how you would reference base Daasity transform scripts located in the Daasity shared and internal repos.

References to script files that exist outside of your custom code repository must use a specific syntax. The syntax can be composed of the following pieces:

  • GIT repository provider: currently only GitHub is supported (github://)

  • Git repository name (e.g.: platform-sql-shared)

  • file folder path (e.g.: /scripts/)

  • file name with the explicit file extension .SQL (e.g.: some_shopify_script_in_the_Daasity_shared_repo.sql)

  • optionally a branch name: this option should be rarely used and should never be used in a production deployment. If a branch is not specified it assumes master

Referencing integrations

Manifest files must be in root folder

Your script manifest files must remain within the root folder of your custom code repository, but your transformation code can be in any folder.

Daasity will maintain script manifest files in the following folder which you can reference to help create your own versions.

: this script manifest file contains all the scripts needed to run the entire workflow and includes the code for every integration that has base code. You can remove the appropriate lines to build your version as well as reference this file to find the most recent additions / changes

: this script manifest file contains all the scripts needed to run the Attribution 2.0 feature

: this script manifest file contains all the scripts needed if you have changed the Attribution Setting in the BSD and need to reset the tables to reflect the new attribution settings

<section_name> - Identifier for a group of scripts. Must be one word, or multiple words in . There can be as many sections in your manifest file as you see fit.

conditional_integrations * - A boolean statement that specifies which integrations must be active and authorized in your Daasity account for the scripts in the current section to run. The statement should be enclosed in single or double quotes. You can use AND and OR logic in this statement, and you can use parentheses to group the statements. For example: conditional_integrations: "google_analytics AND (bigcommerce or shopify)" will make it so that the scripts in the section will run only if there is a Universal Analytics integration and either a Shopify or BigCommerce integration in the account. . The conditional_integrations tag is optional. If you leave it out, all of the scripts in the section will run no matter what.

scripts - A list of SQL script files that will be executed in sequential fashion, top to bottom. Read the section below to help you understand how to reference different types of files.

When adding an integration dependency, you must use a valid integration reference. If you have access to our shared code repo, you can find the full list of valid integration references in .

🔀
https://github.com/Daasity/platform-sql-shared/tree/master/scripts/script_manifest_files
daily_incremental.yml
attribution_2_0.yml
attribution_2_0_reset.yml
snake case
the config/integrations_map.yml file
Learn about using proper integration references here
Referencing script files
workflows
our template daily_incremental.yml file
daily workflow
A snapshot of our template daily_incremntal.yml file