Retrieve AWS Credentials from the Dataiku API

This tutorial explains how to retrieve AWS credentials through the Dataiku Python API. By configuring an S3 connection with AssumeRole, you can obtain temporary STS credentials scoped to individual users and use them to interact with AWS services programmatically.

Prerequisites

• A Dataiku instance with a the permission to configure an S3 connection.

If you already have the S3 connection configured, you can skip the “Connecting AWS and Dataiku” step.

Connecting AWS and Dataiku

Before you can retrieve AWS credentials, you complete the following setup:

1. Ensure the machine running Dataiku has an IAM instance profile.

2. Enable an S3 connection with AssumeRole mode and the Details readable by parameter configured. See the AssumeRole tutorial for guidance.

3. As the role to assume, use ${adminProperty:associatedRole}.

4. In each user’s settings, under Admin Properties, add an entry named associatedRole with the ARN or name of the IAM role to assume for that user.

5. Ensure the IAM instance profile of the machine running Dataiku has permission to assume the required roles.

Retrieving credentials

The following code retrieves AWS credentials using the Dataiku API and the S3 connection configured above.

In DataikuOutside Dataiku

import dataiku

# Replace with the name of your S3 connection
MY_S3_CONNECTION = ""

conn_info = dataiku.api_client().get_connection(MY_S3_CONNECTION).get_info()
cred = conn_info.get_aws_credential()

The returned cred object is a dictionary containing temporary Security Token Service (STS) credentials, including accessKey, secretKey, and sessionToken.

Note

If you run this code on an API endpoint such as a Kubernetes-deployed API node, initialize the Dataiku API client with dataikuapi.DSSClient() before calling get_connection().

Using the credentials with Boto3

With the retrieved credentials, you can create an authenticated Boto3 session to interact with AWS services. The example below lists all S3 buckets accessible to the assumed role.

import boto3

# Create a session using the retrieved credentials
session = boto3.Session(
    aws_access_key_id=cred['accessKey'],
    aws_secret_access_key=cred['secretKey'],
    aws_session_token=cred['sessionToken']
)

# Create an S3 client from the session
s3 = session.client('s3')

# List all S3 buckets accessible to the assumed role
response = s3.list_buckets()
print(response['Buckets'])

You can use the same session object to create clients for other AWS services, such as session.client('sts') or session.client('secretsmanager').

Wrapping up

In this tutorial, you configured an S3 connection with AssumeRole in Dataiku, retrieved temporary AWS credentials through the Python API, and used them to authenticate a boto3 session. This approach scopes AWS access per user based on the associatedRole admin property, making it well-suited for multi-user environments where fine-grained AWS permission management is required.