API for plugin FS providers#

class dataiku.fsprovider.FSProvider(root, config, plugin_config)#

The base interface for a Custom FS provider

Parameters:

  • root – the root path for this provider

  • config – the dict of the configuration of the object

  • plugin_config – contains the plugin settings

close()#

Perform any necessary cleanup

stat(path)#

Get the info about the object at the given path

Parameters:

path – where the object to inspect is located

Returns:

a dict with the fields:

  • ’path’ : the location of the object (relative to the root this instance was created with)

  • ’isDirectory’ : True if the object is a folder

  • ’size’ : size of the object in bytes, 0 for folders

  • ’lastModified’ : modification time in ms since epoch, -1 if not defined

If there is no object at the given location, return None

set_last_modified(path, last_modified)#

Change the modification time of an object.

Parameters:

  • path – where the object to modify is located.

  • last_modidied – timestamp as ms since epoch

Returns:

True if the change was done, False if not or if the operation is not supported

browse(path)#

Enumerate files non-recursively from a path

Parameters:

path – what to enumerate

Returns:

a dict with the fields:

  • ’fullPath’ : the path from the root this instance was created with

  • ’exists’ : True if there is something at path

  • ’directory’ : True if the path denotes a folder

  • ’size’ : the size of the file at path; 0 if it’s a folder

If the object at path is a folder, then there should be a ‘children’ attribute which is a list of dicts with the same fields (without a ‘children’ field for subfolders)

enumerate(prefix, first_non_empty)#

Enumerate files recursively from a path.

Parameters:

  • prefix – where to start the enumeration

  • first_non_empty – if first_non_empty, stop at the first non-empty file.

Returns:

a list of dicts corresponding to the enumerated files (not folders). Each dict is expected to contain these fields:

  • ’path’ : the path relative to the root this instance was created with,

  • ’size’ : size in bytes

  • ’lastModified’ : modification time in ms since epoch (-1 if not defined)

If there is nothing at the prefix, not even empty folders, return None

delete_recursive(path)#

Delete recursively

Parameters:

path – path to the folder or file to remove

move(from_path, to_path)#

Move (rename) an object

Parameters:

  • from_path – where the data to move is located (relative to the root this instance was created with)

  • to_path – the target path for the data

read(path, stream, limit)#

Read data

Parameters:

  • path – where to read the data (relative to the root this instance was created with)

  • stream – a file-like to write the data into

  • limit – if not -1, max number of bytes needed. Any more bytes will be discarded by the backend

write(path, stream)#

Write data

Parameters:

  • path – where to write the data (relative to the root this instance was created with)

  • stream – a file-like to read the data from