LLM Mesh Core

This page groups the native LLM Mesh objects and query/response builders around the main DSSLLM entry point.

Core objects

class dataikuapi.dss.llm.DSSLLM(client, project_key, llm_id)

A handle to interact with a DSS-managed LLM.

Important

Do not create this class directly, use dataikuapi.dss.project.DSSProject.get_llm() instead.

new_completion()

Create a new completion query.

Returns:

A handle on the generated completion query.

Return type:

DSSLLMCompletionQuery

new_completions()

Create a new multi-completion query.

Returns:

A handle on the generated multi-completion query.

Return type:

DSSLLMCompletionsQuery

new_embeddings(text_overflow_mode='FAIL')

Create a new embedding query.

Parameters:

text_overflow_mode (str) – How to handle longer texts than what the model supports. Either ‘TRUNCATE’ or ‘FAIL’.

Returns:

A handle on the generated embeddings query.

Return type:

DSSLLMEmbeddingsQuery

new_images_generation()
new_reranking()

Create a new reranking query.

Returns:

A handle on the generated reranking query.

Return type:

DSSLLMRerankingQuery

as_langchain_llm(**data)

Create a langchain-compatible LLM object for this LLM.

Returns:

A langchain-compatible LLM object.

Return type:

dataikuapi.dss.langchain.llm.DKULLM

as_langchain_chat_model(**data)

Create a langchain-compatible chat LLM object for this LLM.

Returns:

A langchain-compatible LLM object.

Return type:

dataikuapi.dss.langchain.llm.DKUChatModel

as_langchain_embeddings(**data)

Create a langchain-compatible embeddings object for this LLM.

Returns:

A langchain-compatible embeddings object.

Return type:

dataikuapi.dss.langchain.embeddings.DKUEmbeddings

create_conversation(conversation_id=None, end_user_id=None, metadata=None, message=None)

Create a persisted conversation bound to this LLM.

If message is omitted, return the created conversation handle. If it is provided, execute the first turn and return its completion response, with success set to False when LLM execution fails. The created conversation is then available through DSSLLMConversationCompletionResponse.conversation.

Parameters:

  • conversation_id (str) – Identifier for the conversation.

  • end_user_id (str) – End-user identifier associated with the conversation.

  • metadata (dict) – Metadata associated with the conversation.

  • message – First-turn message or messages, as a string, raw user/system chat message dict, or list of those values.

Returns:

The created conversation handle, or the first persisted completion response when message is provided.

Return type:

Union[dataikuapi.dss.llm.DSSLLMConversation, dataikuapi.dss.llm.DSSLLMConversationCompletionResponse]

class dataikuapi.dss.llm.DSSLLMListItem(client, project_key, data)

An item in a list of llms

Important

Do not instantiate this class directly, instead use dataikuapi.dss.project.DSSProject.list_llms().

to_llm()

Convert the current item.

Returns:

A handle for the llm.

Return type:

dataikuapi.dss.llm.DSSLLM

property id
Returns:

The id of the llm.

Return type:

string

property type
Returns:

The type of the LLM

Return type:

string

property description
Returns:

The description of the LLM

Return type:

string

Text generation

class dataikuapi.dss.llm.DSSLLMCompletionQuery(llm)

A handle to interact with a completion query. Completion queries allow you to send a prompt to a DSS-managed LLM and retrieve its response.

Important

Do not create this class directly, use dataikuapi.dss.llm.DSSLLM.new_completion() instead.

property settings
Returns:

The completion query settings.

Return type:

dict

new_guardrail(type)

Start adding a guardrail to the request. You need to configure the returned object, and call add() to actually add it

Return type:

DSSLLMRequestGuardrailBuilder

execute()

Run the completion query and retrieve the LLM response.

Returns:

The LLM response.

Return type:

DSSLLMCompletionResponse

execute_streamed(collect_response=False)

Run the completion query and retrieve the LLM response as streamed chunks.

Parameters:

collect_response (bool) – If True, the streamed chunks are also aggregated into a consolidated DSSLLMCompletionResponse by the returned iterator.

Returns:

An iterator over the LLM response chunks

Return type:

DSSLLMStreamedCompletionChunks

with_dss_agent_tool(dss_agent_tool)

Add a DSS agent tool to the completion query tools setting

Parameters:

dss_agent_tool (dataikuapi.dss.agent_tool.DSSAgentTool) – The DSS Agent Tool to include in the query’s tools setting

new_multipart_message(role='user')

Start adding a multipart-message to the completion query.

Use this to add image parts to the message.

Parameters:

role (str) – The message role. Use system to set the LLM behavior, assistant to store predefined responses, user to provide requests or comments for the LLM to answer to. Defaults to user.

Return type:

DSSLLMCompletionQueryMultipartMessage

new_multipart_tool_output(tool_call_id, role='tool', output='')

Start adding a multipart tool output to the completion query.

Parameters:

  • tool_call_id (str) – The tool call id, as provided by the LLM in the conversation messages.

  • role (str) – The message role. Defaults to tool.

  • output (str) – The tool’s output. Defaults to an empty string.

Return type:

DSSLLMCompletionQueryMultipartToolOutput

with_context(context)
with_json_output(schema=None, strict=None, compatible=None, if_supported=None)

Request the model to generate a valid JSON response, for models that support it.

Note that some models may require you to also explicitly request this in the user or system prompt to use this.

When if_supported=True, it is recommended to describe the expected JSON structure in the user or system prompt. If the model does not support JSON or schema responses through its API, it receives only those prompt instructions.

Parameters:

  • schema (dict) – (optional) If specified, request the model to produce a JSON response that adheres to the provided schema. Support varies across models/providers.

  • strict (bool) – (optional) If a schema is provided, whether to strictly enforce it. Support varies across models/providers.

  • compatible (bool) – (optional) Allow DSS to modify the schema in order to increase compatibility, depending on known limitations of the model/provider. Defaults to automatic.

  • if_supported (bool) – (optional) If True, ignore JSON output when unsupported by the selected model/provider.

with_memory_fragment(memory_fragment)

Add a memory fragment to the completion query.

Parameters:

memory_fragment (dict) – The memory fragment returned by the model on the previous turn.

with_message(message, role='user')

Add a message to the completion query.

Parameters:

  • message (str) – The message text.

  • role (str) – The message role. Use system to set the LLM behavior, assistant to store predefined responses, user to provide requests or comments for the LLM to answer to. Defaults to user.

with_structured_output(model_type, strict=None, compatible=None, if_supported=None)

Instruct the model to generate a response as an instance of a specified Pydantic model.

This functionality depends on with_json_output and normally requires that the model supports JSON output with a schema.

Caution

Structured output support is experimental for locally-running Hugging Face models.

When if_supported=True, it is recommended to describe the structure represented by model_type in the user or system prompt. If the model does not support JSON or schema responses through its API, it receives only those prompt instructions.

Parameters:
with_tool_calls(tool_calls, role='assistant')

Add tool calls to the completion query.

Caution

Tool calls support is experimental for locally-running Hugging Face models.

Parameters:

  • tool_calls (list[dict]) – Calls to tools that the LLM requested to use.

  • role (str) – The message role. Defaults to assistant.

with_tool_output(tool_output, tool_call_id, role='tool')

Add a tool message to the completion query.

Parameters:

  • tool_output (str) – The tool output, as a string.

  • tool_call_id (str) – The tool call id, as provided by the LLM in the conversation messages.

  • role (str) – The message role. Defaults to tool.

with_tool_validation_requests(tool_validation_requests)

Add tool validation requests to the completion query.

Parameters:

tool_validation_requests (list[dict]) – Validation requests for tools that the agent requested to use.

with_tool_validation_response(validation_request_id, validated=True, arguments=None)

Add a tool validation response to the completion query.

Parameters:

  • validation_request_id (str) – The validation request id, as provided by the agent in the conversation messages.

  • validated (bool) – Whether to validate or reject the tool call.

  • arguments (str) – Arguments to use for the tool call (if different from the validation request).

class dataikuapi.dss.llm.DSSLLMCompletionResponse(raw_resp=None, text=None, finish_reason=None, response_parser=None, trace=None, query=None)

A handle to interact with a completion query result.

Important

Do not create this class directly, use dataikuapi.dss.llm.DSSLLMCompletionQuery.execute() or dataikuapi.dss.llm.DSSLLMCompletionsResponse.responses or dataikuapi.dss.llm.DSSLLMStreamedCompletionChunks.response instead.

property json
Returns:

LLM response parsed as a JSON object

property json_lenient
Returns:

LLM response parsed as JSON, allowing extra text before and after the JSON value.

property parsed

Structured LLM response.

Only available when the completion query used with_structured_output().

Returns:

The LLM response deserialized into an instance of the requested Pydantic model.

Return type:

pydantic.BaseModel

property parsed_lenient
Returns:

Structured LLM response, allowing extra text before or after the JSON value.

property success
Returns:

The outcome of the completion query.

Return type:

bool

property text
Returns:

The raw text of the LLM response.

Return type:

Union[str, None]

property tool_calls
Returns:

The tool calls of the LLM response.

Return type:

Union[list, None]

resolve_tool_calls()

Resolve the tool calls in this response.

Returns:

The resolved tool calls.

Return type:

list[dataikuapi.dss.llm.DSSLLMResolvedToolCall]

property tool_validation_requests
Returns:

The tool validation requests of the agent response.

Return type:

Union[list, None]

property memory_fragment
Returns:

Data generated by the model that must be passed back in the next query.

Return type:

Union[dict, None]

property log_probs
Returns:

The log probs of the LLM response.

Return type:

Union[list, None]

property context_upsert
Returns:

The context upsert of the response (only for agents).

Return type:

Union[dict, None]

property artifacts

Get the artifacts generated by the LLM response.

Returns:

The consolidated artifacts, or an empty list if the response contains no artifacts.

Return type:

list[dict]

property sources

Get the sources used to generate the LLM response.

Returns:

The sources associated with the response, or an empty list if the response contains no sources.

Return type:

list[dict]

get_raw()

Get the complete, unprocessed completion response data.

Returns:

The raw completion response.

Return type:

dict

property trace
Returns:

The trace of the completion query if available, None otherwise.

Return type:

Union[dict, None]

property total_usage
prepare_followup()

Prepare a new completion query to follow up on this response, pre-filled with the relevant data from the response.

Returns:

The prepared follow-up completion query.

Return type:

DSSLLMCompletionQuery

class dataikuapi.dss.llm.DSSLLMResolvedToolCall(raw_tool_call, dss_agent_tool=None)

A tool call from a completion response, with resolved DSS Agent Tool (if applicable) and input.

Important

Do not create this class directly, use dataikuapi.dss.llm.DSSLLMCompletionResponse.resolve_tool_calls() instead.

property dss_agent_tool
Returns:

The DSS Agent Tool resolved for this call, or None if unresolved (e.g., not a DSS Agent Tool).

Return type:

Optional[dataikuapi.dss.agent_tool.DSSAgentTool]

property tool_name
Returns:

The tool name requested by the LLM.

Return type:

str

property subtool_name
Returns:

The resolved sub-tool name, or None for a single tool or an unresolved call.

Return type:

Optional[str]

property input
Returns:

The input supplied by the LLM for this tool call.

Return type:

dict

property tool_call_id
Returns:

The ID of this tool call assigned by the LLM provider.

Return type:

str

get_raw()
Returns:

The unmodified tool call data returned in the LLM completion response.

Return type:

dict

run()

Execute the resolved DSS Agent Tool call.

Returns:

The result of running this tool.

Return type:

dict

class dataikuapi.dss.llm.DSSLLMStreamedCompletionChunks(query, collect_response=False)

An iterator over the chunks generated by the execution of a streamed completion query. The streamed chunks are of type DSSLLMStreamedCompletionChunk and DSSLLMStreamedCompletionFooter. When collect_response=True, the streamed chunks are aggregated into a consolidated DSSLLMCompletionResponse.

Important

Do not create this class directly, use dataikuapi.dss.llm.DSSLLMCompletionQuery.execute_streamed() instead.

iter_chunks()
Returns:

An iterator over the LLM response chunks.

Return type:

Iterator[Union[DSSLLMStreamedCompletionChunk, DSSLLMStreamedCompletionFooter]]

property response
Returns:

The consolidated LLM response obtained by the aggregation of all streamed chunks, if collect_response=True. Available only after all chunks have been collected.

Return type:

DSSLLMCompletionResponse

prepare_followup()

Prepare a followup completion query from the consolidated response, pre-filled with the relevant data from the response. Available only when collect_response=True, after all chunks have been collected.

Returns:

The prepared follow-up completion query.

Return type:

DSSLLMCompletionQuery

class dataikuapi.dss.llm.DSSLLMStreamedCompletionChunk(data)

A handle to interact with a streamed completion query chunk.

Important

Do not create this class directly, iterate over a dataikuapi.dss.llm.DSSLLMStreamedCompletionChunks iterator instead to generate the chunks instead.

property type
Returns:

Type of this chunk, either “content” or “event”

Return type:

Literal[“content”, “event”]

property text
Returns:

If this chunk is content and has text, the (partial) text

Return type:

bool

property event_kind
Returns:

If this chunk is an event, its kind

Return type:

str

get_raw()

Get the raw data for this individual streamed completion chunk.

Some fields, such as artifacts, may be split across several chunks. The value returned by this method is not aggregated and may therefore contain only part of an artifact. To obtain consolidated artifacts and sources, use dataikuapi.dss.llm.DSSLLMCompletionQuery.execute_streamed() with collect_response=True, consume the full stream, and access dataikuapi.dss.llm.DSSLLMStreamedCompletionChunks.response.

Returns:

The unprocessed data received for this chunk.

Return type:

dict

class dataikuapi.dss.llm.DSSLLMStreamedCompletionFooter(data)

A handle to interact with a streamed completion query footer.

Important

Do not create this class directly, iterate over a dataikuapi.dss.llm.DSSLLMStreamedCompletionChunks iterator instead to generate the chunks instead.

property type
Returns:

Type of this chunk, to distinguish it from dataikuapi.dss.llm.DSSLLMStreamedCompletionChunk chunks. Can only be “footer”

Return type:

Literal[“footer”]

property trace
Returns:

The trace of the completion query if available, None otherwise.

Return type:

Union[dict, None]

property total_usage
get_raw()

Get the raw data for the streamed completion footer.

The footer is emitted once, at the end of a successful streamed completion, and contains response-level metadata. To obtain the complete response assembled from the chunks and footer, use dataikuapi.dss.llm.DSSLLMCompletionQuery.execute_streamed() with collect_response=True, consume the full stream, and access dataikuapi.dss.llm.DSSLLMStreamedCompletionChunks.response.

Returns:

The unprocessed data received for the footer.

Return type:

dict

Persisted conversations

class dataikuapi.dss.llm.DSSLLMConversationListItem(client, project_key, data)

An item in a list of persisted LLM conversations.

Important

Do not instantiate this class directly, instead use dataikuapi.dss.project.DSSProject.list_llm_conversations().

to_conversation()

Convert the current item.

Returns:

A handle for the conversation.

Return type:

dataikuapi.dss.llm.DSSLLMConversation

property conversation_id
Returns:

The conversation identifier.

Return type:

str

property end_user_id
Returns:

The end-user identifier, if set.

Return type:

Union[str, None]

property archived_at
Returns:

The archive time in milliseconds since the Unix epoch, if archived.

Return type:

Union[int, None]

property archived
Returns:

Whether the conversation is archived.

Return type:

bool

property default_llm_id
Returns:

The default LLM identifier, if set.

Return type:

Union[str, None]

property llm_id
Returns:

Alias for default_llm_id.

Return type:

Union[str, None]

get_raw()

Retrieve the raw list item data.

Return type:

dict

class dataikuapi.dss.llm.DSSLLMConversation(client, project_key, conversation_id, data=None)

A handle to interact with a persisted LLM conversation.

Conversation properties use the latest metadata snapshot loaded on this handle and never perform network requests. Use refresh() or get_raw() to explicitly reload the snapshot after the conversation changes.

Important

Do not create this class directly. Instead, use dataikuapi.dss.project.DSSProject.get_llm_conversation(), dataikuapi.dss.project.DSSProject.create_llm_conversation(), dataikuapi.dss.project.DSSProject.list_llm_conversations() with as_type="objects", or DSSLLM.create_conversation().

get_raw()

Retrieve the conversation properties and refresh the metadata snapshot. Messages are loaded separately with get_messages().

Return type:

dict

refresh()

Refresh the conversation metadata snapshot.

Returns:

This conversation handle.

Return type:

DSSLLMConversation

get_messages(message_id=None, with_threads=False)

Retrieve persisted conversation messages.

By default, include messages of the latest thread (in case of multiple threads in this conversation).

Parameters:

  • message_id (str) – If set, retrieve the parent chain up to this message.

  • with_threads (bool) – If True, include all threads. If message_id is also specified, only include child threads of that message.

Return type:

list[dict]

property end_user_id
Returns:

The end-user identifier, if set.

Return type:

Union[str, None]

property archived_at
Returns:

The archive time in milliseconds since the Unix epoch, if archived.

Return type:

Union[int, None]

property archived
Returns:

Whether the conversation is archived.

Return type:

bool

property default_llm_id
Returns:

The default LLM identifier, if set.

Return type:

Union[str, None]

property llm_id
Returns:

Alias for default_llm_id.

Return type:

Union[str, None]

property metadata
Returns:

The conversation metadata, if set.

Return type:

Union[dict, None]

property last_message_id
Returns:

The latest message identifier, if there is a message.

Return type:

Union[str, None]

property current_context
Returns:

The current conversation context, if set.

Return type:

Union[dict, None]

update(end_user_id=None, archived=None, default_llm_id=None, metadata=None)

Update persisted conversation metadata.

Parameters:

  • end_user_id (str) – Updated end-user identifier.

  • archived (bool) – Archive state.

  • default_llm_id (str) – Updated default LLM identifier.

  • metadata (dict) – Updated conversation metadata.

Returns:

The updated conversation properties, with the same shape as get_raw().

Return type:

dict

delete()

Hard-delete the conversation.

Raises a dataikuapi.utils.DataikuException if deletion fails.

new_completion(parent_message_id=None, llm_id=None)

Prepare a new turn on this persisted conversation.

Parameters:

  • parent_message_id (str) – If set, the new message follows the specified message, otherwise it follows this conversation’s latest message.

  • llm_id (str) – LLM identifier to use for this turn.

Returns:

A persisted conversation completion query.

Return type:

DSSLLMConversationCompletionQuery

class dataikuapi.dss.llm.DSSLLMConversationCompletionQuery(conversation, parent_message_id=None, llm_id=None)

A query that appends a turn to an existing persisted conversation.

Important

Do not create this class directly. Use DSSLLMConversation.new_completion(), DSSLLMConversationCompletionResponse.prepare_followup(), or DSSLLMConversationStreamedCompletionChunks.prepare_followup() instead.

property settings

The completion settings for this persisted conversation turn.

Return type:

dict

new_guardrail(type)

Start adding a guardrail to this persisted conversation turn.

Configure the returned object, then call add() to add it to the turn.

Return type:

DSSLLMRequestGuardrailBuilder

new_multipart_message(role='user')

Start adding a multipart input message to this persisted conversation turn.

Parameters:

role (str) – Must be user or system.

Return type:

DSSLLMCompletionQueryMultipartMessage

with_message(message, role='user')

Add a user or system input message to this persisted conversation turn.

Replayed assistant histories and tool artifacts are not accepted here. :param str message: The message text. :param str role: Must be user or system.

with_context(context)

Not supported for persisted conversations.

Conversation context is stored server-side and updated from persisted turn responses.

with_memory_fragment(memory_fragment)

Not supported for persisted conversations.

Persisted conversations replay stored memory fragments automatically.

with_tool_calls(tool_calls, role='assistant')

Not supported for persisted conversations.

Persisted conversations replay stored assistant tool calls automatically.

with_tool_validation_requests(tool_validation_requests)

Not supported for persisted conversations.

Use with_tool_validation_response() to resume a persisted turn that is waiting for tool validation.

with_tool_validation_response(validation_request_id, validated=True, arguments=None)

Add a tool validation response to resume a pending persisted conversation turn.

Parameters:

  • validation_request_id (str) – The validation request id, as provided by the agent in the persisted conversation messages.

  • validated (bool) – Whether to validate or reject the tool call.

  • arguments (str) – Arguments to use for the tool call, if different from the validation request.

new_multipart_tool_output(tool_call_id, role='tool', output='')

Start adding a multipart tool output to resume this persisted conversation.

Add one output for every tool call returned by the selected parent response before executing the turn.

Parameters:

  • tool_call_id (str) – The tool call id, as provided by the LLM in the persisted conversation response.

  • role (str) – Must be tool.

  • output (str) – The tool’s text output. Defaults to an empty string.

Return type:

DSSLLMCompletionQueryMultipartToolOutput

with_tool_output(tool_output, tool_call_id, role='tool')

Add a tool output to resume this persisted conversation.

Add one output for every tool call returned by the selected parent response before executing the turn. Tool outputs cannot be mixed with a user message or tool validation responses in the same turn.

Parameters:

  • tool_output (str) – The tool output, as a string.

  • tool_call_id (str) – The tool call id, as provided by the LLM in the persisted conversation response.

  • role (str) – Must be tool.

execute()

Append and execute this persisted conversation turn.

LLM execution failures are returned as a persisted response with success set to False.

Returns:

The persisted conversation turn response.

Return type:

DSSLLMConversationCompletionResponse

execute_streamed(collect_response=False)

Append and stream this persisted conversation turn.

Parameters:

collect_response (bool) – If True, the streamed chunks are also aggregated into a consolidated DSSLLMConversationCompletionResponse by the returned iterator.

Returns:

An iterator over the persisted conversation response chunks.

Return type:

DSSLLMConversationStreamedCompletionChunks

with_json_output(schema=None, strict=None, compatible=None, if_supported=None)

Request the model to generate a valid JSON response, for models that support it.

Note that some models may require you to also explicitly request this in the user or system prompt to use this.

When if_supported=True, it is recommended to describe the expected JSON structure in the user or system prompt. If the model does not support JSON or schema responses through its API, it receives only those prompt instructions.

Parameters:

  • schema (dict) – (optional) If specified, request the model to produce a JSON response that adheres to the provided schema. Support varies across models/providers.

  • strict (bool) – (optional) If a schema is provided, whether to strictly enforce it. Support varies across models/providers.

  • compatible (bool) – (optional) Allow DSS to modify the schema in order to increase compatibility, depending on known limitations of the model/provider. Defaults to automatic.

  • if_supported (bool) – (optional) If True, ignore JSON output when unsupported by the selected model/provider.

with_structured_output(model_type, strict=None, compatible=None, if_supported=None)

Instruct the model to generate a response as an instance of a specified Pydantic model.

This functionality depends on with_json_output and normally requires that the model supports JSON output with a schema.

Caution

Structured output support is experimental for locally-running Hugging Face models.

When if_supported=True, it is recommended to describe the structure represented by model_type in the user or system prompt. If the model does not support JSON or schema responses through its API, it receives only those prompt instructions.

Parameters:
class dataikuapi.dss.llm.DSSLLMConversationCompletionResponse(raw_resp, conversation, response_parser=None, query=None)

Response to a persisted conversation turn.

Important

Do not create this class directly. Responses are returned by DSSLLMConversationCompletionQuery.execute(), DSSLLM.create_conversation() when called with message, dataikuapi.dss.project.DSSProject.create_llm_conversation() when called with message, and DSSLLMConversationStreamedCompletionChunks.response.

property conversation_id
Returns:

The persisted conversation identifier.

Return type:

str

property last_message_id
Returns:

The latest persisted message identifier, if available.

Return type:

Union[str, None]

property conversation
Returns:

The persisted conversation handle.

Return type:

DSSLLMConversation

prepare_followup()

Prepare a follow-up turn pinned to this persisted response.

Returns:

The prepared follow-up persisted conversation query.

Return type:

DSSLLMConversationCompletionQuery

class dataikuapi.dss.llm.DSSLLMConversationStreamedCompletionChunks(query, collect_response=False)

Streamed chunks for a persisted conversation turn.

iter_chunks()
Returns:

An iterator over the persisted conversation response chunks.

Return type:

Iterator[Union[DSSLLMStreamedCompletionChunk, DSSLLMStreamedCompletionFooter]]

property response
Returns:

The consolidated persisted conversation response obtained by aggregating all streamed chunks, if collect_response=True. Available only after all chunks have been collected.

Return type:

DSSLLMConversationCompletionResponse

prepare_followup()

Prepare a follow-up turn pinned to the consolidated persisted response.

Available only when collect_response=True, after all chunks have been collected.

Returns:

The prepared follow-up persisted conversation query.

Return type:

DSSLLMConversationCompletionQuery

Batch text generation

class dataikuapi.dss.llm.DSSLLMCompletionsQuery(llm)

A handle to interact with a multi-completion query. Completion queries allow you to send a prompt to a DSS-managed LLM and retrieve its response.

Important

Do not create this class directly, use dataikuapi.dss.llm.DSSLLM.new_completion() instead.

property settings
Returns:

The completion query settings.

Return type:

dict

new_completion()
new_guardrail(type)

Start adding a guardrail to the request. You need to configure the returned object, and call add() to actually add it

Return type:

DSSLLMRequestGuardrailBuilder

execute()

Run the completions query and retrieve the LLM response.

Returns:

The LLM response.

Return type:

DSSLLMCompletionsResponse

with_json_output(schema=None, strict=None, compatible=None, if_supported=None)

Request the model to generate a valid JSON response, for models that support it.

Note that some models may require you to also explicitly request this in the user or system prompt to use this.

When if_supported=True, it is recommended to describe the expected JSON structure in the user or system prompt. If the model does not support JSON or schema responses through its API, it receives only those prompt instructions.

Parameters:

  • schema (dict) – (optional) If specified, request the model to produce a JSON response that adheres to the provided schema. Support varies across models/providers.

  • strict (bool) – (optional) If a schema is provided, whether to strictly enforce it. Support varies across models/providers.

  • compatible (bool) – (optional) Allow DSS to modify the schema in order to increase compatibility, depending on known limitations of the model/provider. Defaults to automatic.

  • if_supported (bool) – (optional) If True, ignore JSON output when unsupported by the selected model/provider.

with_structured_output(model_type, strict=None, compatible=None, if_supported=None)

Instruct the model to generate a response as an instance of a specified Pydantic model.

This functionality depends on with_json_output and normally requires that the model supports JSON output with a schema.

Caution

Structured output support is experimental for locally-running Hugging Face models.

When if_supported=True, it is recommended to describe the structure represented by model_type in the user or system prompt. If the model does not support JSON or schema responses through its API, it receives only those prompt instructions.

Parameters:
class dataikuapi.dss.llm.DSSLLMCompletionsQuerySingleQuery
new_multipart_message(role='user')

Start adding a multipart-message to the completion query.

Use this to add image parts to the message.

Parameters:

role (str) – The message role. Use system to set the LLM behavior, assistant to store predefined responses, user to provide requests or comments for the LLM to answer to. Defaults to user.

Return type:

DSSLLMCompletionQueryMultipartMessage

with_message(message, role='user')

Add a message to the completion query.

Parameters:

  • message (str) – The message text.

  • role (str) – The message role. Use system to set the LLM behavior, assistant to store predefined responses, user to provide requests or comments for the LLM to answer to. Defaults to user.

with_memory_fragment(memory_fragment)

Add a memory fragment to the completion query.

Parameters:

memory_fragment (dict) – The memory fragment returned by the model on the previous turn.

with_tool_calls(tool_calls, role='assistant')

Add tool calls to the completion query.

Caution

Tool calls support is experimental for locally-running Hugging Face models.

Parameters:

  • tool_calls (list[dict]) – Calls to tools that the LLM requested to use.

  • role (str) – The message role. Defaults to assistant.

with_tool_validation_requests(tool_validation_requests)

Add tool validation requests to the completion query.

Parameters:

tool_validation_requests (list[dict]) – Validation requests for tools that the agent requested to use.

with_tool_validation_response(validation_request_id, validated=True, arguments=None)

Add a tool validation response to the completion query.

Parameters:

  • validation_request_id (str) – The validation request id, as provided by the agent in the conversation messages.

  • validated (bool) – Whether to validate or reject the tool call.

  • arguments (str) – Arguments to use for the tool call (if different from the validation request).

new_multipart_tool_output(tool_call_id, role='tool', output='')

Start adding a multipart tool output to the completion query.

Parameters:

  • tool_call_id (str) – The tool call id, as provided by the LLM in the conversation messages.

  • role (str) – The message role. Defaults to tool.

  • output (str) – The tool’s output. Defaults to an empty string.

Return type:

DSSLLMCompletionQueryMultipartToolOutput

with_tool_output(tool_output, tool_call_id, role='tool')

Add a tool message to the completion query.

Parameters:

  • tool_output (str) – The tool output, as a string.

  • tool_call_id (str) – The tool call id, as provided by the LLM in the conversation messages.

  • role (str) – The message role. Defaults to tool.

with_context(context)
class dataikuapi.dss.llm.DSSLLMCompletionsResponse(raw_resp, response_parser=None)

A handle to interact with a multi-completion response.

Important

Do not create this class directly, use dataikuapi.dss.llm.DSSLLMCompletionsQuery.execute() instead.

property responses

The array of responses

Multipart message and tool payload builders

class dataikuapi.dss.llm.DSSLLMCompletionQueryMultipartMessage(q, role)

Important

Do not create this class directly, use dataikuapi.dss.llm.DSSLLMCompletionQuery.new_multipart_message() or dataikuapi.dss.llm.DSSLLMCompletionsQuerySingleQuery.new_multipart_message().

add()

Add this message to the completion query

with_text(text)

Add a text part to the multipart message

Parameters:

text (str) – The text to add

with_inline_image(image, mime_type=None)

Add an image part to the multipart message

Parameters:

  • image (Union[str, bytes]) – The image

  • mime_type (str) – None for default

with_captioned_image_inline(caption, image, mime_type=None)

Add a captioned image part to the multipart message

Parameters:

  • caption (str) – Image caption

  • image (Union[str, bytes]) – The image

  • mime_type (str) – None for default

with_image_url(image)

Add an image url part to the multipart message

Parameters:

image (str) – The image url

class dataikuapi.dss.llm.DSSLLMCompletionQueryMultipartToolOutput(q, tool_call_id, role, output)

Important

Do not create this class directly, use dataikuapi.dss.llm.DSSLLMCompletionQuery.new_multipart_tool_output() or dataikuapi.dss.llm.DSSLLMCompletionsQuerySingleQuery.new_multipart_tool_output().

add()

Add this tool output to the completion query

with_text(text)

Add a text part to the multipart tool output

Parameters:

text (str) – The text to add

with_inline_image(image, mime_type=None)

Add an image part to the multipart tool output

Parameters:

  • image (Union[str, bytes]) – The image

  • mime_type (str) – None for default

with_captioned_image_inline(caption, image, mime_type=None)

Add a captioned image part to the multipart tool output

Parameters:

  • caption (str) – Image caption

  • image (Union[str, bytes]) – The image

  • mime_type (str) – None for default

with_image_url(image)

Add an image url part to the multipart tool output

Parameters:

image (str) – The image url

Guardrails

class dataikuapi.dss.llm.DSSLLMRequestGuardrailBuilder(request, type)

Important

Do not create this class directly. Use dataikuapi.dss.llm.DSSLLMCompletionQuery.new_guardrail(), dataikuapi.dss.llm.DSSLLMCompletionsQuery.new_guardrail(), dataikuapi.dss.llm.DSSLLMConversationCompletionQuery.new_guardrail(), dataikuapi.dss.llm.DSSLLMEmbeddingsQuery.new_guardrail(), or dataikuapi.dss.llm.DSSLLMImageGenerationQuery.new_guardrail().

property params
Returns:

The parameters of this guardrail.

Return type:

dict

add()

Add this guardrail to the completion query.

Embeddings

class dataikuapi.dss.llm.DSSLLMEmbeddingsQuery(llm, text_overflow_mode)

A handle to interact with an embedding query. Embedding queries allow you to transform text into embedding vectors using a DSS-managed model.

Important

Do not create this class directly, use dataikuapi.dss.llm.DSSLLM.new_embeddings() instead.

add_text(text)

Add text to the embedding query.

Parameters:

text (str) – Text to add to the query.

add_image(image, text=None)

Add an image to the embedding query.

Parameters:

  • image – Image content as bytes or str (base64)

  • text – Optional text (requires a multimodal model)

new_guardrail(type)

Start adding a guardrail to the request. You need to configure the returned object, and call add() to actually add it

Return type:

DSSLLMRequestGuardrailBuilder

execute()

Run the embedding query.

Returns:

The results of the embedding query.

Return type:

DSSLLMEmbeddingsResponse

class dataikuapi.dss.llm.DSSLLMEmbeddingsResponse(raw_resp)

A handle to interact with an embedding query result.

Important

Do not create this class directly, use dataikuapi.dss.llm.DSSLLMEmbeddingsQuery.execute() instead.

get_embeddings()

Retrieve vectors resulting from the embeddings query.

Returns:

A list of lists containing all embedding vectors.

Return type:

list

Image generation

class dataikuapi.dss.llm.DSSLLMImageGenerationQuery(llm)

A handle to interact with an image generation query.

Important

Do not create this class directly, use dataikuapi.dss.llm.DSSLLM.new_images_generation() instead.

with_prompt(prompt, weight=None)

Add a prompt to the image generation query.

Parameters:

  • prompt (str) – The prompt text.

  • weight (float) – Optional weight between 0 and 1 for the prompt.

with_negative_prompt(prompt, weight=None)

Add a negative prompt to the image generation query.

Parameters:

  • prompt (str) – The prompt text.

  • weight (float) – Optional weight between 0 and 1 for the negative prompt.

with_original_image(image, mode=None, weight=None)

Add an image to the generation query.

To edit specific pixels of the original image. A mask can be applied by calling with_mask():

>>> query.with_original_image(image, mode="INPAINTING") # replace the pixels using a mask

To edit an image:

>>> query.with_original_image(image, mode="MASK_FREE") # edit the original image according to the prompt

>>> query.with_original_image(image, mode="VARY") # generates a variation of the original image
Parameters:

  • image (Union[str, bytes]) – The original image as str in base 64 or bytes.

  • mode (str) – The edition mode. Modes support varies across models/providers.

  • weight (float) – The original image weight between 0 and 1.

with_mask(mode, image=None)

Add a mask for edition to the generation query. Call this method alongside with_original_image().

To edit parts of the image using a black mask (replace the black pixels):

>>> query.with_mask("MASK_IMAGE_BLACK", image=black_mask)

To edit parts of the image that are transparent (replace the transparent pixels):

>>> query.with_mask("ORIGINAL_IMAGE_ALPHA")
Parameters:

  • mode (str) – The mask mode. Modes support varies across models/providers.

  • image (Union[str, bytes]) – The mask image to apply to the image edition. As str in base 64 or bytes.

new_guardrail(type)

Start adding a guardrail to the request. You need to configure the returned object, and call add() to actually add it

Return type:

DSSLLMRequestGuardrailBuilder

property height
Returns:

The generated image height in pixels.

Return type:

Optional[int]

property width
Returns:

The generated image width in pixels.

Return type:

Optional[int]

property fidelity
Returns:

From 0.0 to 1.0, how strongly to adhere to prompt.

Return type:

Optional[float]

property quality
Returns:

Quality of the image to generate. Valid values depend on the targeted model.

Return type:

Optional[str]

property seed
Returns:

Seed of the image to generate, gives deterministic results when set.

Return type:

Optional[int]

property style
Returns:

Style of the image to generate. Valid values depend on the targeted model.

Return type:

Optional[str]

property images_to_generate
Returns:

Number of images to generate per query. Valid values depend on the targeted model.

Return type:

Optional[int]

property aspect_ratio
Returns:

The width/height aspect ratio or None if either is not set.

Return type:

Optional[float]

execute()

Executes the image generation

Return type:

DSSLLMImageGenerationResponse

class dataikuapi.dss.llm.DSSLLMImageGenerationResponse(raw_resp)

A handle to interact with an image generation response.

Important

Do not create this class directly, use dataikuapi.dss.llm.DSSLLMImageGenerationQuery.execute() instead.

property success
Returns:

The outcome of the image generation query.

Return type:

bool

first_image(as_type='bytes')
Parameters:

as_type (str) – The type of image to return, ‘bytes’ for bytes otherwise ‘str’ for base 64 str.

Returns:

The first generated image as bytes or str depending on the as_type parameter.

Return type:

Union[bytes,str]

get_images(as_type='bytes')
Parameters:

as_type (str) – The type of images to return, ‘bytes’ for bytes otherwise ‘str’ for base 64 str.

Returns:

The generated images as bytes or str depending on the as_type parameter.

Return type:

Union[List[bytes], List[str]]

property images
Returns:

The generated images in bytes format.

Return type:

List[bytes]

property trace
Returns:

The trace of the image generation query if available, None otherwise.

Return type:

Union[dict, None]

property total_usage

Reranking

class dataikuapi.dss.llm.DSSLLMRerankingQuery(llm)

A handle to interact with a reranking query. Reranking queries allow you to send a text query and a list of documents to a DSS-managed ranking model and retrieve the documents ranked according to their relevance to the query.

Important

Do not create this class directly, use dataikuapi.dss.llm.DSSLLM.new_reranking() instead.

with_query(text)

Sets the reranking text query.

Parameters:

text (str) – The reranking text query.

with_document(text)

Adds a text document to the list of documents to be reranked.

Parameters:

text (str) – The text document to be reranked.

execute()

Run the reranking query and retrieve the LLM response.

Returns:

The LLM response.

Return type:

DSSLLMRerankingResponse

class dataikuapi.dss.llm.DSSLLMRerankingResponse(raw_resp)

A handle to interact with a ranking query result.

Important

Do not create this class directly, use dataikuapi.dss.llm.DSSLLMRerankingQuery.execute() instead.

property success
Returns:

The outcome of the reranking query.

Return type:

bool

property error_message
Returns:

The error message if the reranking query failed, None otherwise.

Return type:

Union[str, None]

property documents
Returns:

The array of reranked documents.

Return type:

list of DSSLLMRerankingResponse.RankedDocument

property trace
Returns:

The trace of the reranking query if available, None otherwise.

Return type:

Union[dict, None]

class RankedDocument(raw_doc)
property index
Returns:

The index of the document in the original request.

Return type:

int

property relevance_score
Returns:

The relevance score assigned to the document by the ranking model.

Return type:

float