Scenarios (in a scenario)

This is the documentation of the API for use in scenarios.

Warning

This API can only be used within a scenario in order to run steps and report on progress of the current scenario.

If you want to control scenarios, please see Scenarios

These functions can be used both for “Execute Python code” steps in steps-based scenarios, and for full Python scenarios

A quick description of Python scenarios can be found in Definitions. More details and usage samples are also available in Custom scenarios

The Scenario is the main class you’ll use to interact with DSS in your “Execute Python code” steps and Python scenarios.

Detailed examples

Set Scenario step timeout

There is no explicit timeout functionality for a Build step within a Scenario. A common question is how to setup a timeout for a scenario or scenario step to avoid situations where a scenario gets stuck/hung in a running state indefinitely.

You can implement it using the Dataiku Python API. The same scenario step can be re-written as a custom Python step, in which case you can add additional Python code to implement a timeout.

Here is a code sample that you can try:

###########################################################################################
# !! CUSTOM SCENARIO EXAMPLE !!                                                           #
# See https://doc.dataiku.com/dss/latest/scenarios/custom_scenarios.html for more details #
###########################################################################################

import time
import dataiku
from dataiku.scenario import Scenario, BuildFlowItemsStepDefHelper
from dataikuapi.dss.future import DSSFuture

TIMEOUT_SECONDS = 3600

s = Scenario()

# Replace this commented block by your Scenario steps
# Example: build a Dataset
step_handle = s.build_dataset("your_dataset_name", asynchronous=True)

start = time.time()
while not step_handle.is_done():
    end = time.time()
    print("Duration: {}s".format(end-start))
    if end - start > TIMEOUT_SECONDS:
        f = DSSFuture(dataiku.api_client(), step_handle.future_id)
        f.abort()
        raise Exception("Scenario was aborted because it took too much time.")

Reference documentation

dataiku.scenario.scenario.Scenario()	Handle to the current (running) scenario.
dataiku.scenario.scenario.BuildFlowItemsStepDefHelper(...)	Helper to build the definition of a 'Build Flow Items' step.