Querying the LLM from an headless API

In this tutorial, you will learn how to build an API from a web application’s backend (called headless API) and how to use it from code. You will use the LLM Mesh to query an LLM and serving the response.

You can use a headless web application to create an API endpoint for a particular purpose that doesn’t fit well in the API Node. For example, you may encounter this need if you want to use an LLM, access datasets, etc. You should use this functionality if you can’t use the API Node. The API Node is the preferred way to expose functionalities as it is scalable and highly available.

Prerequisites

StandardDash

• Dataiku >= 13.1

Building the server

The first step is to define the routes you want your API to handle. A single route is responsible for a (simple) process. Dataiku provides an easy way to describe those routes. Relying on a Flask server helps you return the desired resource types. Check the API access in the web apps’ settings to use this functionality, as shown in Figure 1.

Figure 1: Enabling API access.

This tutorial relies on a single route (query) to query an LLM. As the message sent to the LLM could be very long, you can not consider passing the message in the URL. So, you will have to send the message in the body of the request. For this, GET method is not the recommended way, as it is not suppose to have a body. You will use the POST method to send the message in a JSON body associated to the request. Once the query comes to the server, the server will extract the message, send it to the LLM and return the response to the user.

Before accessing to the body, you have to check if the Content-type is well defined. Then, you must extract the message from the body. Send the message to the LLM, and if querying the LLM is successful return the response to the user. Highlighted lines in Code1 show how to do. The remaining lines show how to set up the whole application.

StandardDash

Code 1: LLM Headless API

import dataiku
from flask import request, make_response

LLM_ID = "openai:openai:gpt-3.5-turbo"
llm = dataiku.api_client().get_default_project().get_llm(LLM_ID)


@app.route('/query', methods=['POST'])
def query():
    content_type = request.headers.get('Content-Type')
    if content_type == 'application/json':
        json = request.json
        user_message = json.get('message', None)
        if user_message:
            completion = llm.new_completion()
            completion.with_message(user_message)
            resp = completion.execute()

            if resp.success:
                msg = resp.text
            else:
                msg = "Something went wrong"
        else:
            msg = "No message was found"

        response = make_response(msg)
        response.headers['Content-type'] = 'application/json'
        return response
    else:
        return 'Content-Type is not supported!'

Querying the API

To access the headless API, you must be logged on to the instance or have an API key that identifies you. If you need help setting up an API key, please read this tutorial. Then, there are several different ways to interact with a headless API.

cUrlPython

Using cUrl requires an API key to access the headless API or an equivalent way of authenticating, depending on the authentication method set on the Dataiku instance. Once you have this API key, you can access the API endpoint with the following command. The WEBAPP_ID is the first eight characters (before the underscore) in the webapp URL. For example, if the webapp URL in DSS is /projects/HEADLESS/webapps/kUDF1mQ_api/view, the WEBAPP_ID is kUDF1mQ and the PROJECT_KEY is HEADLESS.

Code 2: cUrl command to fetch data

curl -X POST --header 'Authorization: Bearer <USE_YOUR_API_KEY>' \
 'http://<DSS_ADDRESS>:<DSS_PORT>/web-apps-backends/<PROJECT_KEY>/<WEBAPP_ID>/query' \
 --header 'Content-Type: application/json' \
 --data '{
     "message": "Write an haiku on boardgames"
 }'

Wrapping up

If you need to give access to unauthenticated users, you can turn your web application into a public one, as this documentation suggests. Now that you understand how to turn a web application into a headless one, you can create an agent-headless API.