Local Models For LLM Mesh

For the overall proposed structure, see LLM Mesh.

Local model objects

class dataikuapi.dss.local_model.DSSLocalModel(client, connection_name, model_id)

A local model defined in a DSS connection.

Important

Do not instantiate this class directly, use dataikuapi.dss.admin.DSSConnection.get_local_model() instead.

property model_id
get_status()

Fetch the current model status.

Returns:

Model status wrapper.

Return type:

DSSLocalModelStatus

wake_up()

Request wakeup of a model kernel.

shutdown(force=False)

Request shutdown of model kernels.

Parameters:

force (boolean) – If True, force shutdown.

as_llm(project_key=None)

Return this model as a DSS LLM handle.

Parameters:

project_key (string) – Project key to bind the LLM handle to. If None, uses the client’s default project.

Returns:

DSS LLM Handle for this local model.

Return type:

DSSLLM

get_settings()

Fetch the model settings from the connection definition.

Returns:

Model settings wrapper.

Return type:

DSSHFModelSettings

class dataikuapi.dss.local_model.DSSLocalModelStatus(client, connection_name, data)

Status of a local model.

Important

Do not instantiate this class directly, use dataikuapi.dss.local_model.DSSLocalModel.get_status() instead.

property model_id
property kernels

Per-kernel statuses for this model.

Returns:

List of kernel statuses.

Return type:

list[DSSLocalModelKernel]

property state

Aggregated model state.

Returns:

one of READY, STARTING, STOPPING, ERROR, STOPPED

Return type:

string

get_raw()

Get the raw API payload for this model status.

Returns:

Raw model status dictionary.

Return type:

dict

class dataikuapi.dss.local_model.DSSLocalModelKernel(client, connection_name, data)

Status of a single local model kernel.

Important

Do not instantiate this class directly, use dataikuapi.dss.local_model.DSSLocalModelStatus.kernels instead.

property kernel_id
property model_id
property state

Current state of the kernel.

Returns:

one of READY, STARTING, STOPPING, ERROR, STOPPED

Return type:

string

property metrics
Returns:

Kernel metrics if available (if kernel state is READY)

Return type:

dict | None

get_log()

Fetch logs for this specific model kernel.

Returns:

Log payload

Return type:

dict

get_raw()

Get the raw API payload for this kernel status.

Returns:

Raw kernel status dictionary.

Return type:

dict

class dataikuapi.dss.local_model.DSSHFModelSettings(client, connection_name, data)

Settings of a HuggingFace local model.

Important

Do not instantiate this class directly, use dataikuapi.dss.local_model.DSSLocalModel.get_settings() instead.

property model_id
get_raw()

Get the raw model settings payload.

Returns:

Raw model settings dictionary.

Return type:

dict