Workflow Metrics

This article provides an overview of the different Workflow metrics captured by Daasity and how to use the different metric pages to get additional detail on the different components of the workflow

Overview

Watch this video to see a demo of the different metrics pages available to get an overview of your overall execution time as well as details on the extraction, load and transformation sections of each workflow:

There are seven (7) different charts or logs that provide metrics on workflows:

• Workflow Execution Timeline Chart

• Workflow Last Execution Detail Chart

• Workflow Execution Runtimes Chart

• Workflow Execution Detail Log

• Extraction Detail Log

• Data Loading Detail Log

• Transformation Detail Log

Workflow Execution Timeline Chart

From the Workflow page you can find the Workflow Execution Timeline chart which shows the timeline for all the workflows that were started in the last 24 hours. You can use the slider at the bottom of the chart or the buttons at the top of the chart to zoom in or out of the timeline

Workflow Last Execution Detail Chart

From the workflow page you can find the Workflow Last Execution Detail chart by navigating to the bottom of the page to the Configured Workflows section and then hovering over the workflow you want to drill into and clicking on the metrics icon as show below.

This will expand the workflow to show additional detail on the specific workflow with execution time for the extraction, load and transform steps (where applicable)

Workflow Execution Runtimes Chart

From the Configured Workflows section on the Workflows page you can access the Workflow Execution Runtimes chart by clicking on the name of the workflow. This will open the Workflows Execution Runtimes page that will show:

1. The runtime for each time this workflow was executed both in the chart and in the table

2. The average runtime for the period selected which is shown as a dotted red line in the chart

Workflow Execution Detail Log

From either the Workflow Execution Timeline Chart or the Workflow Execution Runtimes Chart you can get to the Workflow Execution Detail Log that will show you details on the duration for each step of the workflow both at a summary level but also at a detail level

• Total duration and status of the workflow

• Duration of the extraction, data loading and transformation steps of the workflow

• Duration for each extraction

Extraction Detail Log

From the Workflow Execution Detail Log you can click on a specific extraction to show details on all the tables that were extracted and the number of rows extracted for each table

Data Loading Detail Log

From the Workflow Execution Detail Log you can click on Data Loading to open the Data Loading Detail Log which will show the total time to load all tables and the time to load each table

Transformation Detail Log

From the Workflow Execution Detail Log you can click on Transformation to open the Transformation Detail Log which will show the list of each transformation SQL script that was run as part of this step and the time to execute this specific script.

This log can be very helpful to identify scripts that are long running