You are viewing the developer guide for version 13 of DSS.

Plugins#

class dataikuapi.dss.plugin.DSSPlugin(client, plugin_id)#

A plugin on the DSS instance.

Important

Do not instantiate directly, use dataikuapi.DSSClient.get_plugin()

get_settings()#

Get the plugin-level settings.

Note

This call requires an API key with either:

Returns:: a handle on the settings
Return type:: DSSPluginSettings

get_project_settings(project_key)#

Get the project-level settings.

Note

This call requires an API key with either:

Returns:: a handle on the project-level settings
Return type:: DSSPluginProjectSettings

create_code_env(python_interpreter=None, conda=False)#

Start the creation of the code env of the plugin.

Note

This call requires an API key with either:

If not passing any value to python_interpreter, the default defined by the plugin will be used.

Usage example:

# create a default code env for the plugin
plugin = client.get_plugin('the-plugin-id')
future = plugin.create_code_env()
creation = future.wait_for_result()
# take the name of the new code env
env_name = creation["envName"]
# set it as the current plugin code env
settings = plugin.get_settings()
settings.set_code_env(env_name)
settings.save()

Parameters:

python_interpreter (string) – which version of python to use. Possible values: PYTHON27, PYTHON34, PYTHON35, PYTHON36, PYTHON37, PYTHON38, PYTHON39, PYTHON310, PYTHON311
conda (boolean) – if True use conda to create the code env, if False use virtualenv and pip.

Returns:

a handle on the operation

Return type:

dataikuapi.dss.future.DSSFuture

update_code_env()#

Start an update of the code env of the plugin.

Note

This call requires an API key with either:

Usage example:

# update the plugin code env after updating the plugin
plugin = client.get_plugin('the-plugin-id')
future = plugin.update_code_env()
future.wait_for_result()

Returns:: a handle on the operation
Return type:: dataikuapi.dss.future.DSSFuture

update_from_zip(fp)#

Update the plugin from a plugin archive (as a file object).

Note

This call requires an API key with either:

Parameters:: fp (file-like) – A file-like object pointing to a plugin archive zip

start_update_from_zip(fp)#

Update the plugin from a plugin archive (as a file object). Returns immediately with a future representing the process done asynchronously

Note

This call requires an API key with either:

Parameters:: fp (file-like) – A file-like object pointing to a plugin archive zip
Returns:: A DSSFuture representing the update process
Return type:: DSSFuture

update_from_store()#

Update the plugin from the Dataiku plugin store.

Note

This call requires an API key with either:

Usage example:

# update a plugin that was installed from the store
plugin = client.get_plugin("my-plugin-id")
future = plugin.update_from_store()
future.wait_for_result()

Returns:: a handle on the operation
Return type:: dataikuapi.dss.future.DSSFuture

update_from_git(repository_url, checkout='master', subpath=None)#

Updates the plugin from a Git repository.

Note

DSS must be setup to allow access to the repository.

Note

This call requires an API key with either:

Usage example:

# update a plugin that was installed from git
plugin = client.get_plugin("my-plugin-id") 
future = plugin.update_from_git("git@github.com:myorg/myrepo")
future.wait_for_result()

Parameters:

repository_url (string) – URL of a Git remote
checkout (string) – branch/tag/SHA1 to commit. For example “master”
subpath (string) – Optional, path within the repository to use as plugin. Should contain a ‘plugin.json’ file

Returns:

a handle on the operation

Return type:

dataikuapi.dss.future.DSSFuture

list_usages(project_key=None)#

Get the list of usages of the plugin.

Note

This call requires an API key with either:

Parameters:: project_key (string) – optional key of project where to look for usages. Default is None and looking in all projects.
Return type:: DSSPluginUsages

delete(force=False)#

Delete a plugin.

Note

This call requires an API key with either:

Parameters:: force (boolean) – if True, plugin will be deleted even if usages are found or errors occurred during usages analysis. Default: False.
Return type:: dataikuapi.dss.future.DSSFuture

list_files()#

Get the hierarchy of files in the plugin.

Note

Dev plugins only

Note

This call requires an API key with either:

Returns:: list of files or directories, each one a dict. Directories have a children field for recursion. Each dict has fields name and path (the path from the root of the plugin files)
Return type:: dict

get_file(path)#

Get a file from the plugin folder.

Note

Dev plugins only

Note

This call requires an API key with either:

Usage example:

# read the code env desc of a plugin
plugin = client.get_plugin("my-plugin-name")
with plugin.get_file('code-env/python/desc.json') as fp:
    desc = json.load(fp)

Parameters:: path (string) – the path of the file, relative to the root of the plugin
Returns:: the file’s content
Return type:: file-like

put_file(path, f)#

Update a file in the plugin folder.

Note

Dev plugins only

Note

This call requires an API key with either:

Parameters:

f (file-like) – the file contents, as a file-like object
path (string) – the path of the file, relative ot the root of the plugin

rename_file(path, new_name)#

Rename a file/folder in the plugin.

Note

Dev plugins only

Parameters:

path (string) – the path of the file/folder, relative ot the root of the plugin
new_name (string) – the new name of the file/folder

move_file(path, new_path)#

Move a file/folder in the plugin.

Note

Dev plugins only

Note

This call requires an API key with either:

Parameters:

path (string) – the path of the file/folder, relative ot the root of the plugin
new_path (string) – the new path relative at the root of the plugin

class dataikuapi.dss.plugin.DSSPluginSettings(client, plugin_id, settings)#

The settings of a plugin.

Important

Do not instantiate directly, use DSSPlugin.get_settings().

set_code_env(code_env_name)#

Set the name of the code env to use for this plugin.

Parameters:: code_env_name (string) – name of a code env

list_parameter_sets()#

List the parameter sets defined in this plugin.

Return type:: list[DSSPluginParameterSet]

get_parameter_set(parameter_set_name)#

Get a parameter set in this plugin.

Parameters:: parameter_set_name (string) – name of the parameter set
Returns:: a handle on the parameter set
Return type:: DSSPluginParameterSet

get_raw()#

Get the raw settings object.

Note

This method returns a reference to the settings, not a copy. Changing values in the reference then calling save() results in these changes being saved.

Returns:: the settings as a dict. The instance-level settings consist of the plugin code env’s name,

the presets and the permissions to use the plugin components. The project-level settings consist of the presets and the parameter set descriptions.

Return type:: dict

list_parameter_set_names()#

List the names of the parameter sets defined in this plugin.

Return type:: list[string]

save()#: Save the settings to DSS.

class dataikuapi.dss.plugin.DSSPluginProjectSettings(client, plugin_id, settings, project_key)#

The project-level settings of a plugin.

Important

Do not instantiate directly, use DSSPlugin.get_project_settings().

start_save()#

Save the settings to DSS. Returns with a future representing the post actions done asynchronously (e.g. rebuild cde image for visual recipes)

Returns:: A DSSFuture representing the save post process
Return type:: DSSFuture

list_parameter_sets()#

List the parameter sets defined in this plugin.

Return type:: list[DSSPluginProjectParameterSet]

get_parameter_set(parameter_set_name)#

Get a parameter set in this plugin.

Parameters:: parameter_set_name (string) – name of the parameter set
Returns:: a handle on the parameter set
Return type:: DSSPluginProjectParameterSet

get_raw()#

Get the raw settings object.

Note

This method returns a reference to the settings, not a copy. Changing values in the reference then calling save() results in these changes being saved.

Returns:: the settings as a dict. The instance-level settings consist of the plugin code env’s name,

the presets and the permissions to use the plugin components. The project-level settings consist of the presets and the parameter set descriptions.

Return type:: dict

list_parameter_set_names()#

List the names of the parameter sets defined in this plugin.

Return type:: list[string]

save()#: Save the settings to DSS.

class dataikuapi.dss.plugin.DSSPluginParameterSet(plugin_settings, desc, settings, presets)#

A parameter set in a plugin.

Important

Do not instantiate directly, use DSSPluginSettings.get_parameter_set() or DSSPluginSettings.list_parameter_sets().

The values in this class can be modified directly, and changes will be taken into account when calling DSSPluginSettings.save()

property definable_inline#

Whether presets for this parameter set can be defined directly in the form of the datasets, recipes, …

Return type:: bool

property definable_at_project_level#

Whether presets for this parameter set can be defined at the project level

Return type:: bool

create_preset(preset_name, with_defaults=False)#

Create a new preset of this parameter set in the plugin settings.

Parameters:

preset_name (string) – name for the preset to create
with_defaults (bool) – if True, fill the new preset with the default value for each parameter

Returns:

a preset definition, as a DSSPluginPreset (see get_preset())

Return type:

dict

delete_preset(preset_name)#

Remove a preset from this plugin’s settings

Parameters:: preset_name (string) – name for the preset to remove

property desc#

Get the raw definition of the parameter set.

Returns:: a parameter set definition, as a dict. The parameter set’s contents is a desc sub-dict. See the doc
Return type:: dict

get_preset(preset_name)#

Get a preset of this parameter set.

Parameters:: preset_name (string) – name of a preset
Returns:: a handle on the preset definition, or None if the preset doesn’t exist
Return type:: DSSPluginPreset

list_preset_names()#

List the names of the presets of this parameter set.

Return type:: list[string]

list_presets()#

List the presets of this parameter set.

Return type:: list[DSSPluginPreset]

save()#: Save the settings to DSS.

property settings#

Get the settings of the parameter set.

These settings control the behavior of the parameter set, and comprise notably the permissions, but not the presets of this parameter set.

Returns:: the settings of the parameter set, as a dict. The parameter set’s settings consist of the permissions controlling whether the presets of the parameter set can be created inline or at the project level.
Return type:: dict

class dataikuapi.dss.plugin.DSSPluginProjectParameterSet(plugin_settings, desc, settings, presets)#

A parameter set in a plugin.

Important

Do not instantiate directly, use DSSPluginProjectSettings.get_parameter_set() or DSSPluginProjectSettings.list_parameter_sets()

The values in this class can be modified directly, and changes will be taken into account when calling or DSSPluginProjectSettings.save()

create_preset(preset_name, with_defaults=False)#

Create a new preset of this parameter set in the plugin settings.

Parameters:

preset_name (string) – name for the preset to create
with_defaults (bool) – if True, fill the new preset with the default value for each parameter

Returns:

a preset definition, as a DSSPluginPreset (see get_preset())

Return type:

dict

delete_preset(preset_name)#

Remove a preset from this plugin’s settings

Parameters:: preset_name (string) – name for the preset to remove

property desc#

Get the raw definition of the parameter set.

Returns:

a parameter set definition, as a dict. The parameter set’s contents is a desc sub-dict. See the doc

Return type:

dict

get_preset(preset_name)#

Get a preset of this parameter set.

Parameters:: preset_name (string) – name of a preset
Returns:: a handle on the preset definition, or None if the preset doesn’t exist
Return type:: DSSPluginPreset

list_preset_names()#

List the names of the presets of this parameter set.

Return type:: list[string]

list_presets()#

List the presets of this parameter set.

Return type:: list[DSSPluginPreset]

save()#: Save the settings to DSS.

property settings#

Get the settings of the parameter set.

These settings control the behavior of the parameter set, and comprise notably the permissions, but not the presets of this parameter set.

Returns:: the settings of the parameter set, as a dict. The parameter set’s settings consist of the permissions controlling whether the presets of the parameter set can be created inline or at the project level.
Return type:: dict

class dataikuapi.dss.plugin.DSSPluginPreset(plugin_settings, settings, parameter_set_desc)#

A preset of a parameter set in a plugin.

Important

Do not instantiate directly, use DSSPluginParameterSet.get_preset() , DSSPluginParameterSet.list_presets() or DSSPluginParameterSet.create_preset(). For project-level presets, use DSSPluginProjectParameterSet.get_preset() , DSSPluginProjectParameterSet.list_presets() or DSSPluginProjectParameterSet.create_preset()

The values in this class can be modified directly, and changes will be taken into account when calling DSSPluginSettings.save().

get_raw()#

Get the raw settings of the preset object.

Note

This method returns a reference to the preset, not a copy. Changing values in the reference then calling save() results in these changes being saved.

Returns:: the preset’s complete settings
Return type:: dict

property name#

Get the name of the preset.

Returns:: the name of the preset
Return type:: string

property config#

Get the raw config of the preset object.

Note

This method returns a reference to the preset, not a copy. Changing values in the reference then calling save() results in these changes being saved.

Returns:: the preset’s config as a dict. Each parameter of the parameter set is a field in the dict.
Return type:: dict

property plugin_config#

Get the raw admin-level config of the preset object. Admin-level config parameters are not shown in the UI to non-admin users.

Note

This method returns a reference to the preset, not a copy. Changing values in the reference then calling save() results in these changes being saved.

Returns:: the preset’s admin config as a dict. Each parameter of the parameter set is a field in the dict.
Return type:: dict

property owner#

The DSS user that owns this preset

Return type:: string

property usable_by_all#

Whether the preset is usable by any DSS user

Return type:: bool

get_permission_item(group)#

Get permissions on the preset for a given group

Parameters:: group (string) – the name of the DSS group you want to check permissions for.
Returns:: the permissions as a dict
Return type:: dict

is_usable_by(group)#

Get whether the preset is usable by DSS users in a group

Parameters:: group (string) – the name of the DSS group you want to check permissions for.
Returns:: True if the preset can be used by DSS users belonging to group. If group is None then returns True if the preset can be used by any DSS user (like usable_by_all())
Return type:: bool

set_usable_by(group, use)#

Set whether the preset is usable by DSS users in a group

Parameters:

group (string) – the name of the DSS group you want to change permissions for.
use (bool) – whether the group should be allowed to use the preset or not

save()#: Save the settings to DSS.

class dataikuapi.dss.plugin.DSSPluginUsage(data)#

Information on a usage of an element of a plugin.

Important

Do no instantiate directly, use dataikuapi.dss.plugin.DSSPlugin.list_usages()

property element_kind#

Get the type of the plugin component.

Returns:: a kind of plugin component, like ‘python-clusters’, ‘python-connectors’, ‘python-fs-providers’, ‘webapps’, …
Return type:: string

property element_type#

Get the identifier of the plugin component.

Return type:: string

property object_id#

Get the identifier of the object using the plugin component.

Return type:: string

property object_type#

Get the type of the object using the plugin component.

Returns:: a type of DSS object, like ‘CLUSTER’, ‘DATASET’, ‘RECIPE’, …
Return type:: string

property project_key#

Get the project key of the object using the plugin component.

Return type:: string

class dataikuapi.dss.plugin.DSSMissingType(data)#

Information on a type not found while analyzing usages of a plugin.

Missing types can occur when plugins stop defining a given component, for example during development, and DSS object still use the now-removed component.

Important

Do no instantiate directly, use dataikuapi.dss.plugin.DSSPlugin.list_usages()

property missing_type#

Get the type of the plugin component.

Return type:: string

property object_id#

Get the identifier of the object using the plugin component.

Return type:: string

property object_type#

Get the type of the object using the plugin component.

Returns:: a type of DSS object, like ‘CLUSTER’, ‘DATASET’, ‘RECIPE’, …
Return type:: string

property project_key#

Get the project key of the object using the plugin component

Return type:: string

class dataikuapi.dss.plugin.DSSPluginUsages(data)#

Information on the usages of a plugin.

Important

Do no instantiate directly, use dataikuapi.dss.plugin.DSSPlugin.list_usages()

Some custom types may not be found during usage analysis, typically when a plugin was removed but is still used. This prevents some detailed analysis and may hide some uses. This information is provided in missing_types().

get_raw()#

Get the raw plugin usages.

Returns:: a summary of the usages, as a dict with fields usages and missingTypes.
Return type:: dict

property usages#

Get the list of usages of components of the plugin.

Returns:: list of usages, each a DSSPluginUsage
Return type:: list

property missing_types#

Get the list of usages of missing components of the plugin.

Returns:: list of missing component types, each a DSSMissingType
Return type:: list

maybe_used()#

Whether the plugin maybe in use.

Returns:: True if usages were found, False if errors were encountered during analysis
Return type:: boolean

class dataikuapi.dss.macro.DSSMacro(client, project_key, runnable_type, definition=None)#

A macro on the DSS instance.

Important

Do not instantiate directly, use dataikuapi.dss.project.DSSProject.get_macro()

get_definition()#

Get the macro definition.

Note

The adminParams field is empty unless the authentication of the API client covers admin rights.

Returns:: the definition (read-only), as a dict. The fields mimic the contents of the runnable.json file of the macro.
Return type:: dict

run(params=None, admin_params=None, wait=True)#

Run the macro from the project

Note

If the authentication of the api client does not have admin rights, admin params are ignored.

Usage example:

# list all datasets on a connection.            
connection_name = 'filesystem_managed'
macro = project.get_macro('pyrunnable_builtin-macros_list-datasets-using-connection')
run_id = macro.run(params={'connection': connection_name}, wait=True)
# the result of this builtin macro is of type RESULT_TABLE
result = macro.get_result(run_id, as_type="json")
for record in result["records"]:
    print("Used by %s" % record[0])

Parameters:

params (dict) – parameters to the macro run (defaults to {})
admin_params (dict) – admin parameters to the macro run (defaults to {})
wait (boolean) – if True, the call blocks until the run is finished

Returns:

a run identifier to use with abort(), get_status() and get_result()

Return type:

string

abort(run_id)#

Abort a run of the macro.

Parameters:: run_id (string) – a run identifier, as returned by run()

get_status(run_id)#

Poll the status of a run of the macro.

Note

Once the run is done, when get_result() is called, the run ceases to exist. Afterwards get_status() will answer that the run doesn’t exist.

Parameters:: run_id (string) – a run identifier, as returned by run()
Returns:: the status, as a dict. Whether the run is still ongoing can be assessed with the running field.
Return type:: dict

get_result(run_id, as_type=None)#

Retrieve the result of a run of the macro.

Note

If the macro is still running, an Exception is raised.

The type of the contents of the result to expect can be checked using get_definition(), in particular the “resultType” field.

Parameters:

run_id (string) – a run identifier, as returned by run() method
as_type (string) – if not None, one of ‘string’ or ‘json’. Use ‘json’ when the type of result advertised by the macro is RESULT_TABLE or JSON_OBJECT.

Returns:

the contents of the result of the macro run, as a file-like is as_type is None; as a str if as_type is “string”; as an object if as_type is “json”.

Return type:

file-like, or string