Jobs#

For usage information and examples, see Jobs

class dataikuapi.dss.project.JobDefinitionBuilder(project, job_type='NON_RECURSIVE_FORCED_BUILD')#

Helper to run a job. Do not create this class directly, use DSSProject.new_job()

with_type(job_type)#

Sets the build type

Parameters:

job_type – the build type for the job RECURSIVE_BUILD, NON_RECURSIVE_FORCED_BUILD, RECURSIVE_FORCED_BUILD, RECURSIVE_MISSING_ONLY_BUILD

with_refresh_metastore(refresh_metastore)#

Sets whether the hive tables built by the job should have their definitions refreshed after the corresponding dataset is built

Parameters:

refresh_metastore (bool) –

with_output(name, object_type=None, object_project_key=None, partition=None)#

Adds an item to build in this job

Parameters:

  • name – name of the output object

  • object_type – type of object to build from: DATASET, MANAGED_FOLDER, SAVED_MODEL, STREAMING_ENDPOINT (defaults to None)

  • object_project_key – PROJECT_KEY for the project that contains the object to build (defaults to None)

  • partition – specify partition to build (defaults to None)

get_definition()#

Gets the internal definition for this job

start()#

Starts the job, and return a dataikuapi.dss.job.DSSJob handle to interact with it.

You need to wait for the returned job to complete

Returns:

a job handle

Return type:

dataikuapi.dss.job.DSSJob

start_and_wait(no_fail=False)#

Starts the job, waits for it to complete and returns a dataikuapi.dss.job.DSSJob handle to interact with it

Raises if the job failed.

Parameters:

no_fail – if True, does not raise if the job failed (defaults to False).

Returns:

A job handle

Return type:

dataikuapi.dss.job.DSSJob

class dataikuapi.dss.job.DSSJob(client, project_key, id)#

A job on the DSS instance.

Important

Do not instantiate this class directly, instead use dataikuapi.dss.project.DSSProject.get_job() or dataikuapi.dss.project.DSSProject.start_job().

abort()#

Aborts the job.

Returns:

a confirmation message for the request

Return type:

dict

get_status()#

Gets the current status of the job.

Returns:

the state of the job

Return type:

dict

get_log(activity=None)#

Gets the logs of the job. If an activity is passed in the parameters the logs will be scoped to that activity.

Parameters:

activity (string) – (optional) the name of the activity in the job whose log is requested (defaults to None)

Returns:

the job logs

Return type:

string

class dataikuapi.dss.job.DSSJobWaiter(job)#

Creates a helper to wait for the completion of a job.

Parameters:

job (dataikuapi.dss.job.DSSJob) – The job to wait for

wait(no_fail=False)#

Waits for the job to finish. If the job fails or is aborted, an exception is raised unless the no_fail parameter is set to True.

Parameters:

no_fail (boolean) – (optional) should an error be raised if the job finished with another status than DONE (defaults to False)

Raises:

DataikuException – when the job does not complete successfully

Returns:

the job state

Return type:

dict