Create a tumbling window trigger dependency

APPLIES TO: Azure Data Factory Azure Synapse Analytics

This article provides steps to create a dependency on a tumbling window trigger. For general information about tumbling window triggers, see Create a tumbling window trigger.

To build a dependency chain and make sure that a trigger is executed only after the successful execution of another trigger within the service, use this advanced feature to create a tumbling window dependency.

For a demonstration on how to create dependent pipelines by using a tumbling window trigger, watch the following video:

Create a dependency in the UI

To create dependency on a trigger, select Trigger > Advanced > New. Then choose the trigger to depend on with the appropriate offset and size. Select Finish and publish the changes for the dependencies to take effect.

Tumbling window dependency properties

A tumbling window trigger with a dependency has the following properties:

{
    "name": "MyTriggerName",
    "properties": {
        "type": "TumblingWindowTrigger",
        "runtimeState": <<Started/Stopped/Disabled - readonly>>,
        "typeProperties": {
            "frequency": <<Minute/Hour>>,
            "interval": <<int>>,
            "startTime": <<datetime>>,
            "endTime": <<datetime - optional>>,
            "delay": <<timespan - optional>>,
            "maxConcurrency": <<int>> (required, max allowed: 50),
            "retryPolicy": {
                "count": <<int - optional, default: 0>>,
                "intervalInSeconds": <<int>>,
            },
            "dependsOn": [
                {
                    "type": "TumblingWindowTriggerDependencyReference",
                    "size": <<timespan - optional>>,
                    "offset": <<timespan - optional>>,
                    "referenceTrigger": {
                        "referenceName": "MyTumblingWindowDependency1",
                        "type": "TriggerReference"
                    }
                },
                {
                    "type": "SelfDependencyTumblingWindowTriggerReference",
                    "size": <<timespan - optional>>,
                    "offset": <<timespan>>
                }
            ]
        }
    }
}

The following table provides the list of attributes needed to define a tumbling window dependency.

Tumbling window self-dependency properties

In scenarios where the trigger shouldn't proceed to the next window until the preceding window is successfully completed, build a self-dependency. A self-dependency trigger that's dependent on the success of earlier runs of itself within the preceding hour has the properties indicated in the following code.

{
    "name": "DemoSelfDependency",
    "properties": {
        "runtimeState": "Started",
        "pipeline": {
            "pipelineReference": {
                "referenceName": "Demo",
                "type": "PipelineReference"
            }
        },
        "type": "TumblingWindowTrigger",
        "typeProperties": {
            "frequency": "Hour",
            "interval": 1,
            "startTime": "2018-10-04T00:00:00Z",
            "delay": "00:01:00",
            "maxConcurrency": 50,
            "retryPolicy": {
                "intervalInSeconds": 30
            },
            "dependsOn": [
                {
                    "type": "SelfDependencyTumblingWindowTriggerReference",
                    "size": "01:00:00",
                    "offset": "-01:00:00"
                }
            ]
        }
    }
}

Usage scenarios and examples

The following scenarios show the use of tumbling window dependency properties. In these examples, the yellow windows need to complete successfully before the next trigger can fire, and the dashed lines connect them to the window that will fire once they complete.

Dependency offset

In these examples, the 10-11 run for trigger A depends on a run from trigger B (highlighted in yellow.) When trigger B successfully completes, trigger A will start.

This dependancy can be concurrent (they can be at the same time), or they can be offset (trigger B that runs at a different time than trigger A.)

In the first example, the 10-11 timeframe trigger A has a concurrent dependancy on the 10-11 trigger B, so there's an offset of 0.

In the second example, the 10-11 timeframe trigger A has an offset dependancy on the 9-10 trigger B, so there's an offset of -1 hour.

Dependency size

In these examples, the 10-11 run for trigger A depends on a run from trigger B (highlighted in yellow.) When trigger B successfully completes, trigger A will start.

The size of the dependancy window can be different than the size of the dependant window.

In the first example, the 10-11 timeframe trigger A has an offset dependancy on the 9-10 trigger B. The offset is -1 hour, and the size is unspecified because the size of the dependancy window is the same size as the trigger window.

In the second example, the 10-11 timeframe trigger A has an offset dependancy on the 9-11 trigger B, so there's an offset of -1 hour and a size of 2 hours.

Self-dependency

In these examples, the 10-11 run for trigger A depends on the previous runs for trigger A. When the specified windows from trigger A successfully complete, the next A trigger will begin.

In the first example, the 10-11 timeframe trigger A has an offset dependancy on the 9-10 trigger A, so there's an offset of -1 hour and a size of unspecified because the size of the dependancy window is the same size as the trigger window.

In the second example, the 10-11 timeframe trigger A has an offset dependancy on the 8-10 trigger A, so there's an offset of -1 hour and a size of 2 hours.

This third example shows a daily job (rather than an hourly job) with no gaps in the output streams of the job.

The 10/2-10/3 timeframe trigger has a dependancy on the 10/1-10/2 window, and will trigger only when 10/1-10/2 completes successfully. The offset is -1 day and the size is unspecified, since the size of the dependancy window is the same size as the trigger window.

Dependency on another tumbling window trigger

The following example shows a daily telemetry processing job that depends on another daily job aggregating the last seven days of output and generates seven-day rolling window streams.

When trigger B's 10/1-10/8 windows successfully complete, trigger A's 10/7-10/8 trigger will start. Trigger A's dependancy has an offset of -6 days, and a size of 7 days.

Monitor dependencies

You can monitor the dependency chain and the corresponding windows from the trigger run monitoring page. Go to Monitoring > Trigger Runs. If a tumbling window trigger has dependencies, the trigger name bears a hyperlink to a dependency monitoring view.

Click through the trigger name to view trigger dependencies. The pane on the right shows trigger run information such as the run ID, window time, and status.

You can see the status of the dependencies and windows for each dependent trigger. If one of the dependencies triggers fails, you must successfully rerun it for the dependent trigger to run.

A tumbling window trigger waits on dependencies for seven days before timing out. After seven days, the trigger run fails.

For a more visual way to view the trigger dependency schedule, select the Gantt view.

Transparent boxes show the dependency windows for each downstream-dependent trigger. Solid-colored boxes shown in the preceding image show individual window runs. Here are some tips for interpreting the Gantt chart view:

• Transparent boxes render blue when dependent windows are in a Pending or Running state.
• After all windows succeed for a dependent trigger, the transparent box turns green.
• Transparent boxes render red when a dependent window fails. Look for a solid red box to identify the failure window run.

To rerun a window in the Gantt chart view, select the solid color box for the window. An action pane pops up with information and rerun options.

Related content

• Create a tumbling window trigger