Skip to content

Large Language Model (LLM)

This module contains functions to manipulate Large Language Model (LLM) prompts and perform LLM completions against the builtin OpenAPI integration and other LLM providers. See https://developer.dialox.ai/platform/nlp/llm/ for more information on how to work with LLM completions in general.

LLM.complete(prompt, bindings \\ %{}, opts \\ [])

Perform a LLM completion request

The full result of the LLM.complete call is a map array which contains the following:

  • text - The output text that LLM produced
  • json - A JSON deserialized version of the text; the runtime detects whether JSON is available in the result and, if so, parses it. The JSON message itself can be padded with arbitrary other texts.
  • usage - The total tokens that were used for this API call
  • request_time - The nr of milliseconds this request took
  • raw - The raw OpenAPI response

LLM.count_tokens(prompt_or_model, text)

Count the number of tokens given a prompt or model and an input text.

Returns nil in case of error

LLM.decode_tokens(prompt_or_model, tokens)

Decode the given list of (integer) tokens into a binary string.

Returns nil in case of error.

LLM.encode_tokens(prompt_or_model, text)

Encode the given text into a list of tokens.

Returns nil in case of error

LLM.prompt(orig \\ %Prompt{}, args)

Construct a new prompt to be used in the LLM completion functions.

This function can be used to override certain properties of other prompts, or create new prompts altogether. However, prefer to use the builtins Prompts YAML files to define prompts, as this is more efficient and less error prone.

To set for instance an extra parameter on a prompt you could do the following:

myprompt = LLM.prompt(@prompts.original, endpoint_params: %{temperature: 1.3})

LLM.render_prompt(prompt, bindings \\ %{})

Render a prompt into its final form, an array of chat messages.

This function is mostly used for debugging and should not be needed in normal use cases.