Hide navigation sidebar

Hide table of contents sidebar

Toggle site navigation sidebar

Dataiku Developer Guide

Toggle table of contents sidebar

Academy
- Join the Academy
  
  Benefit from guided learning opportunities →
Community
Documentation
- Reference Documentation
  
  Comprehensive specifications of Dataiku →
Knowledge
- Knowledge Base
  
  Articles and tutorials on Dataiku features →
Developer
- Developer Guide
  
  Tutorials and articles for developers and coder users →

Getting started
Toggle navigation of Getting started
- Introduction
- Development environment
- Basic workflow
- MLOps lifecycle
- The Dataiku Python APIs
- Quickstart Tutorial
  Toggle navigation of Quickstart Tutorial
Concepts and examples
Toggle navigation of Concepts and examples
- Datasets
  Toggle navigation of Datasets
  - Datasets (reading and writing data)
  - Datasets (other operations)
- Managed folders
- The main API client
- Pyspark recipes
- Projects
- Recipes
- Scenarios
- Flow creation and management
- LLM Mesh
- Visual Machine learning
- Experiment Tracking
- Statistics worksheets
- API Designer & Deployer
- Project Deployer
- Static insights
- Jobs
- Authentication information and impersonation
- Importing tables as datasets
- Wikis
- Discussions
- Performing SQL, Hive and Impala queries
- SQL Query
- Project libraries
- Meanings
- Users and groups
- Connections
- Scenarios (in a scenario)
- Code envs
- Plugins
- Project folders
- Dataiku Applications
- Feature Store
- Streaming Endpoints
- Metrics and checks
- Model Evaluation Stores
- Administration
- Utilities
- API for plugin components
  Toggle navigation of API for plugin components
- Clusters
- Code Studios
- Fleet Manager
  Toggle navigation of Fleet Manager
- Govern
  Toggle navigation of Govern
  - The main GovernClient class
  - Govern Blueprints and Blueprint versions
  - Govern Artifacts
  - Govern Uploaded Files
  - Govern Time series
  - Govern Artifact Search
  - Govern Blueprint Designer
    Toggle navigation of Govern Blueprint Designer
    - Govern Hooks
  - Govern Roles and Permissions
    Toggle navigation of Govern Roles and Permissions
    - Govern Role Assignment Rules
  - Authentication information and impersonation
  - Users and groups
  - Other administration tasks
  - Govern Custom Pages
  - Govern Custom Pages Handler
  - Utilities
- Workspaces
- Data Collections
- Data Quality
- Webapps
Tutorials
Toggle navigation of Tutorials
- Administration
  Toggle navigation of Administration
  - Retrieve AWS Credentials from Dataiku API
- Data Engineering
  Toggle navigation of Data Engineering
- Developer tools
  Toggle navigation of Developer tools
- Generative AI
  Toggle navigation of Generative AI
- Machine Learning
  Toggle navigation of Machine Learning
  - Experiment Tracking
    Toggle navigation of Experiment Tracking
  - Pre-trained Models
    Toggle navigation of Pre-trained Models
  - Model Import
    Toggle navigation of Model Import
    - Importing serialized scikit-learn pipelines as Saved Models for MLOps
  - Model Export
    Toggle navigation of Model Export
    - Wrapping an exported model in a CLI tool
- Plugins development
  Toggle navigation of Plugins development
  - Foreword
  - Creating and configuring a plugin
  - Setting up a dedicated instance for developing plugins
  - Versioning for plugin
    Toggle navigation of Versioning for plugin
    - Git integration for plugins
    - Plugin version management with Git
  - Datasets
    Toggle navigation of Datasets
    - Creating a plugin Dataset component
  - File Format
    Toggle navigation of File Format
    - Writing a plugin File Format component to allow ICal import in Dataiku
  - Macros
    Toggle navigation of Macros
  - Processor plugin component
    Toggle navigation of Processor plugin component
    - Creating a plugin Processor component
  - Prediction algorithm
    Toggle navigation of Prediction algorithm
    - Creating a plugin Prediction Algorithm component
  - Recipes
    Toggle navigation of Recipes
    - Creating a plugin Recipe component
    - Writing a custom recipe to remove outliers from a dataset
  - Webapps
    Toggle navigation of Webapps
    - Turning an existing webapp into a webapp plugin component
- Webapps
  Toggle navigation of Webapps
  - Common subjects on webapps
    Toggle navigation of Common subjects on webapps
    - Impersonation with webapps
    - Accessible resources from webapps
  - Standard (HTML, CSS, JS)
    Toggle navigation of Standard (HTML, CSS, JS)
  - Bokeh
    Toggle navigation of Bokeh
    - Bokeh: Your first webapp
  - Dash
    Toggle navigation of Dash
  - Gradio
    Toggle navigation of Gradio
    - Gradio: your first web application
    - Creating a Gradio application using an LLM-based agent
  - Streamlit
    Toggle navigation of Streamlit
    - Streamlit: your first webapp
  - Use your own framework.
    Toggle navigation of Use your own framework.
API Reference
Toggle navigation of API Reference
- Python
  Toggle navigation of Python

You are viewing the developer guide for version 13 of DSS.

Api Reference
Python
Interaction With Pyspark

Toggle table of contents sidebar

Interaction with Pyspark#

The main API client

Copyright © 2024, Dataiku

Made with Sphinx and @pradyunsg's Furo