Using Dash and LLM Mesh to build a GPT-powered web app assistant#

Prerequisites#

Dataiku >= 13.0
“Use” permission on a code environment using Python >= 3.9 with the following packages:
- dash (tested with version 2.10.2)
- dash-bootstrap-components (tested with version 1.4.1)
Access to an existing project with the following permissions:
- “Read project content”
- “Write project content”

Introduction#

In this tutorial, you will learn to call a GPT model into a Dataiku web app for a simple question-answering task.

LLM initialization and library import#

To begin with, you need to set up a development environment by importing some necessary libraries and initializing the chat LLM you want to use. The tutorial relies on the LLM Mesh for this.

Tip

The documentation provides instructions on obtaining an LLM ID. The following code snippet will print you an exhaustive list of all the models your project has access to.

Code 1: List accessible LLM#

import dataiku
client = dataiku.api_client()
project = client.get_default_project()
llm_list = project.list_llms()
for llm in llm_list:
    print(f"- {llm.description} (id: {llm.id})")

Using the prompt through the LLM Mesh#

To check if everything is working as expected, you can run Code 2 in a notebook.

Code 2: Code for testing if all requirements are met.#

LLM_ID = "" # Replace with a valid LLM id

# Get a text generation model
llm = project.get_llm(LLM_ID)

# Create and run a completion query
completion = llm.new_completion()
completion.with_message("Write a haiku on GPT models")
resp = completion.execute()

# Display the LLM output
if resp.success:
   print(resp.text)

Building the web app#

CSS file#

Before sketching the web app, you must create an accessible CSS file. Go to the Application menu > Global Shared Code > Static web resources. Create a file named loading-state.css inside the local-static directory, with the content shown in Code 3. As shown in the next section, this code will help display an indicator when you query the model.

Code 3: CSS code for indication of a loading state#

*[data-dash-is-loading="true"]{
    visibility: hidden;
}
*[data-dash-is-loading="true"]::before{
    content: "Thinking...";
    display: inline-block;
    visibility: visible;
}

Sketching the Webapp#

First, you need to create an empty Dash webapp. If you don’t know how to create one, please refer to this mini-tutorial.

Then, import all the required libraries and configure the webapp to use the CSS file you created. Code 4 shows how to do this.

Code 4: Import packages#

import dash
from dash import html
from dash import dcc
import dash_bootstrap_components as dbc
from dash.dependencies import Input
from dash.dependencies import Output
from dash.dependencies import State
from dash.exceptions import PreventUpdate
import dataiku
import base64
import pandas as pd

# use the style of examples on the Plotly documentation
app.config.external_stylesheets = [dbc.themes.BOOTSTRAP, "/local/static/loading-state.css"]

Now, you can design the application. You will need a user input and a chatGPT output for this application. As the webapp focuses on a question-answering bot, you should keep some context (the previously asked questions and answers). You must limit the context size to avoid long queries and reduce costs. There are various ways to do that, but the most understandable is restricting the messages kept by a certain amount. Usually, OpenAI uses a token counter to do that, but it is not a human-readable metric. You also may need a button to reset the conversation if the user needs it.

To sum up, we need the following:

a number input (for the size of the kept messages)
a button to reset the conversation
a text input (for the user input)
a text to display the response.

Code 5 implements this application. The highlighted line is where the application will display a processing spinner when the user sends its request. This line works in conjunction with the CSS defined earlier.

Code 5: Design of the application.#

search_text_layout = html.Div([
    dcc.Store(id='messages', data=[
        {"role": "system", "content": "You are a helpful assistant"}]),
    dbc.Row([
        dbc.Label("Max messages", html_for="max_messages", width=2),
        dbc.Col(dbc.Input(id="max_messages", value="5", type="number", min=1, max=10), width=2),
        dbc.Col(width=6),
        dbc.Col(dbc.Button("Reset conversation", id="flush_messages", n_clicks=0, class_name="btn-danger"), width=2,
                class_name="d-grid col-2 gap-2")
    ], class_name="mb-3", ),
    dbc.Row([
        dbc.Label("Ask your question", html_for="search_input", width=2),
        dbc.Col(html.Div(children=[
            dbc.Input(id="search_input", placeholder="What can I do for you?"),
            dcc.Loading(id="ls-loading-1", children=[html.Div(id="ls-loading-output-1")], type="default")]), width=10),
    ], className="mb-3", ),
])

# build your Dash app
app.layout = html.Div([
    search_text_layout,
    dbc.Row([
        dbc.Col(width=2),
        dbc.Col(dbc.Textarea(id="text_output", style={"height": "200px"}), width=10)], class_name="mb-3"),
], className="container-fluid mt-3")

Now that you have designed the application, you only have to connect the components and call the associated functions. The first highlighted line in Code 6 is the callback associated with the Q&A processing. The second one resets the current conversation.

Code 6: Callbacks of the webapp#

@app.callback(
    [Output("ls-loading-output-1", "children"),
     Output("text_output", "value"),
     Output("messages", "data")],
    Input("search_input", "n_submit"),
    State("search_input", "value"),
    State("max_messages", "value"),
    State("messages", "data"),
    running=[
        (Output("search_button", "disabled"), True, False),
    ],
    prevent_initial_call=True
)
def get_answer(_, question, max_messages, messages):
    """
    Ask a question to Chat GPT (with some context), and give back the response
    Args:
        _: number of enter pressed in the input text (not used)
        question: the question (with the context)
        max_messages: number of context messages to keep
        messages: the context

    Returns:
        the response, and an updated version of the context
    """
    if not (question) or not (max_messages) or not (messages):
        raise PreventUpdate

    while len(messages) > int(max_messages):
        messages.pop(1)

    messages.append({"role": "user", "content": question})
    try:
        completion = llm.new_completion()
        for message in messages:
            completion.with_message(message.get('content'), role=message.get('role'))
        answer = completion.execute()

        if answer.success:
            messages.append({"role": "assistant", "content": answer.text})
            return ["", answer.text, messages]
        else:
            return ["", "Something went wrong", messages]
    except:
        return ["", "Something went wrong", messages]


@app.callback(
    Output("messages", "data", allow_duplicate=True),
    Input("flush_messages", "n_clicks"),
    prevent_initial_call=True
)
def reset_conversation(_clicks):
    """
    Reset the conversation
    Args:
        _clicks: number of clicks on the flush button (unused)

    Returns:
        a new context for the conversation
    """
    return [{"role": "system", "content": "You are a helpful assistant"}]

Saving the response into a dataset#

Requesting GPT assistance might be costly, so you could create your cache and save the question that has already been answered and its response. So, before requesting an answer from the GPT assistant, check if the question has been asked. If so, reply with the previous response or ask the GTP assistant if the question has yet to be asked. Depending on the dataset type, adding data to an existing dataset could be done in various ways. Let’s consider two kinds of datasets:

For an SQL-like dataset, you can then use the dataiku.SQLExecutor2 to insert data into a dataset.
For a CSV-like dataset, you can then load the data as a dataframe, add data and save back the dataframe. This method requires loading the whole dataset into memory and may be inappropriate for big datasets.

You first need to create one to save the answers in a dataset. This dataset must have been defined before, with two columns named question and answer.

Adding a button to save the data#

Before digging into the details, you should add a button to the webapp allowing the user to save the question and the answer into a dataset. This is done in two steps:

First, add a save button to the layout:

Code 7: Adding a button for saving the Q&A#

dbc.Row([dbc.Col(dbc.Button("Save this answer",
                            id="save_answer",
                            n_clicks=0,
                            class_name="btn-primary",
                            size="lg"))],
        justify="end",
        className="d-grid gap-2 col-12 mx-auto", )

Then connect this button to a callback in charge of saving the Q&A:

Code 8: Connecting the button to a callback#

@app.callback(
    Output("save_answer", "n_clicks"),
    Input("save_answer", "n_clicks"),
    State("search_input", "value"),
    State("text_output", "value"),
    prevent_initial_call = True
)
def save_answer(_, question, answer):

Depending on the dataset kind, the implementation of the callback will change.

Using the SQLExecutor#

Considering you already have an SQL dataset, you can use the SQLExecutor to insert a row into a dataset, letting the dataset engine optimize for you. To insert a new row, you will need to use an INSERT statement, like sql = f"""INSERT INTO "{table_name}" (question, answer) VALUES {values_string}""". You may face trouble using this kind of statement if the answer contains some specific characters (like ‘, “, …). As you have no control of the response (and fine-tuning the prompt might be tough to prevent the assistant from generating an appropriate answer), you must encode the response before inserting it into the dataset into a proper format.

Code 10 shows how to encode the question and the response before inserting the data using the SQL statement and the dataiku.SQLExecutor2.

Code 9: Saving the data into a SQL-Like dataset#

def write_question_answer_sql(client, dataset_name, connection, connection_type, project_key, question, answer):
    dataset = dataiku.Dataset(dataset_name)
    table_name = dataset.get_location_info().get('info', {}).get('table')
    value_string = (f"('{base64.b64encode(question.encode('utf-8')).decode('utf-8')}', "
                    f"'{base64.b64encode(answer.encode('utf-8')).decode('utf-8')}')")
    sql = f"""INSERT INTO "{table_name}" (question, answer) VALUES {value_string}"""
    client.sql_query(sql, connection=connection, type=connection_type, project_key=dataset.project_key,
                     post_queries=['COMMIT'])

Using dataframe#

If your dataset fits into memory, you can rely on the dataframe to append the data to an existing CSV-like dataset. The principle is straightforward:

Read the dataset as a dataframe.
Create the data.
Append them to the dataframe.
Save back the dataframe.

Code 10 shows how to do this.

Code 10: Saving the data into a CSV-Like dataset#

def write_question_answer_csv(dataset_name, question, answer):
    dataset = dataiku.Dataset(dataset_name)
    row = {
        "question": [f"""{question}"""],
        "answer": [f"""{answer}"""]
    }
    df = dataset.get_dataframe()
    df = pd.concat([df, pd.DataFrame(row)])
    with dataset.get_writer() as writer:
        writer.write_dataframe(df)

Note

Dataiku offers an optional response cache in the LLM Mesh connections, but it keeps data for 24 hours, with a 1GB limit.

Wrapping up#

Congratulations! You have completed this tutorial and built a Dash web application that enables ChatGPT integration inside a Dataiku webapp. Understanding all these basic concepts allows you to create more complex applications.

You can add a field to this web application to tweak the way Chat GPT answers or try to reduce the number of tokens used in a query. If you save the data into a dataset, you can look for the question before requesting the ChatGPT assistant.

In this web application, you keep a fixed number of messages. When this number is reached, you remove the first message. Keeping only a specified number of messages in the conversation is not the best idea. You should rather implement a mechanism that sums up the conversation and keep this summary as the content of the first message. An LLM can do this for you.

Here is the complete code for your application:

webapp.py

import dash
from dash import html
from dash import dcc
import dash_bootstrap_components as dbc
from dash.dependencies import Input
from dash.dependencies import Output
from dash.dependencies import State
from dash.exceptions import PreventUpdate
import dataiku
import base64
import pandas as pd


def write_question_answer_sql(client, dataset_name, connection, connection_type, project_key, question, answer):
    """
    Save data into a SQL like dataset
    Args:
        client: the dataiku client
        dataset_name: name of the SQL dataset
        connection: name of the SQL connection used for saving
        connection_type: type of connection
        project_key: project key
        question: the question
        answer: the answer
    """
    dataset = dataiku.Dataset(dataset_name)
    table_name = dataset.get_location_info().get('info', {}).get('table')
    value_string = f"('{base64.b64encode(question.encode('utf-8')).decode('utf-8')}', '{base64.b64encode(answer.encode('utf-8')).decode('utf-8')}')"
    sql = f"""INSERT INTO "{table_name}" (question, answer) VALUES {value_string}"""
    client.sql_query(sql, connection=connection, type=connection_type, project_key=dataset.project_key,
                     post_queries=['COMMIT'])


def write_question_answer_csv(dataset_name, question, answer):
    """
    Save data into a CSV like dataset
    Args:
        dataset_name: the CSV dataset
        question: the question
        answer: the answer
    """
    dataset = dataiku.Dataset(dataset_name)
    row = {
        "question": [f"""{question}"""],
        "answer": [f"""{answer}"""]
    }
    df = dataset.get_dataframe()
    df = pd.concat([df, pd.DataFrame(row)])
    with dataset.get_writer() as writer:
        writer.write_dataframe(df)


# use the style of examples on the Plotly documentation
app.config.external_stylesheets = [dbc.themes.BOOTSTRAP, "/local/static/loading-state.css"]

search_text_layout = html.Div([
    dcc.Store(id='messages', data=[
        {"role": "system", "content": "You are a helpful assistant"}]),
    dbc.Row([
        dbc.Label("Max messages", html_for="max_messages", width=2),
        dbc.Col(dbc.Input(id="max_messages", value="5", type="number", min=1, max=10), width=2),
        dbc.Col(width=6),
        dbc.Col(dbc.Button("Reset conversation", id="flush_messages", n_clicks=0, class_name="btn-danger"), width=2,
                class_name="d-grid col-2 gap-2")
    ], class_name="mb-3", ),
    dbc.Row([
        dbc.Label("Ask your question", html_for="search_input", width=2),
        dbc.Col(html.Div(children=[
            dbc.Input(id="search_input", placeholder="What can I do for you?"),
            dcc.Loading(id="ls-loading-1", children=[html.Div(id="ls-loading-output-1")], type="default")]), width=10),
    ], className="mb-3", ),
])

# build your Dash app
app.layout = html.Div([
    search_text_layout,
    dbc.Row([
        dbc.Col(width=2),
        dbc.Col(dbc.Textarea(id="text_output", style={"height": "200px"}), width=10)], class_name="mb-3"),
    dbc.Row(
        [dbc.Col(dbc.Button("Save this answer", id="save_answer", n_clicks=0, class_name="btn-primary", size="lg"))],
        justify="end", className="d-grid gap-2 col-12 mx-auto", )
], className="container-fluid mt-3")

LLM_ID = "openai:toto:gpt-3.5-turbo"
client = dataiku.api_client()
project = client.get_default_project()
llm = project.get_llm(LLM_ID)


@app.callback(
    [Output("ls-loading-output-1", "children"),
     Output("text_output", "value"),
     Output("messages", "data")],
    Input("search_input", "n_submit"),
    State("search_input", "value"),
    State("max_messages", "value"),
    State("messages", "data"),
    running=[
        (Output("search_button", "disabled"), True, False),
    ],
    prevent_initial_call=True
)
def get_answer(_, question, max_messages, messages):
    """
    Ask a question to Chat GPT (with some context), and give back the response
    Args:
        _: number of enter pressed in the input text (not used)
        question: the question (with the context)
        max_messages: number of context messages to keep
        messages: the context

    Returns:
        the response, and an updated version of the context
    """
    if not (question) or not (max_messages) or not (messages):
        raise PreventUpdate

    while len(messages) > int(max_messages):
        messages.pop(1)

    messages.append({"role": "user", "content": question})
    try:
        completion = llm.new_completion()
        for message in messages:
            completion.with_message(message.get('content'), role=message.get('role'))
        answer = completion.execute()

        if answer.success:
            messages.append({"role": "assistant", "content": answer.text})
            return ["", answer.text, messages]
        else:
            return ["", "Something went wrong", messages]
    except:
        return ["", "Something went wrong", messages]


@app.callback(
    Output("messages", "data", allow_duplicate=True),
    Input("flush_messages", "n_clicks"),
    prevent_initial_call=True
)
def reset_conversation(_clicks):
    """
    Reset the conversation
    Args:
        _clicks: number of clicks on the flush button (unused)

    Returns:
        a new context for the conversation
    """
    return [{"role": "system", "content": "You are a helpful assistant"}]


@app.callback(
    Output("save_answer", "n_clicks"),
    Input("save_answer", "n_clicks"),
    State("search_input", "value"),
    State("text_output", "value"),
    prevent_initial_call=True
)
def save_answer(_clicks, question, answer):
    """
    Save the answer
    Args:
        _clicks: number of clicks on the flush button (unused)
        question: the question
        answer: the answer

    Returns:

    """
    ## Uncomment these lines if you need to save into an SQL dataset

    #    client =  dataiku.api_client()
    #dataset_name = "History_SQL"
    #connection = "PostgreSQL"
    #connection_type = "sql"
    #project_key = client.get_default_project()

    #write_question_answer_sql(client, dataset_name, connection, connection_type, project_key, question, answer)

    ## Saving into a CSV dataset

    dataset_name = "History"
    write_question_answer_csv(dataset_name, question, answer)