Skip to content

LLM Knowledge base support

It is possible to use a knowledge base to improve the accuracy of the LLM's responses. Most knowledge bases are configured in the prompts file, under the knowledge key; the Google Vertex AI RAG knowledge bases can only be created in the AI section of the studio instead.

Each knowledge base has its own retrieval strategy. Depending on the retrieval strategy, the knowledge base will function differently.

Configuring a knowledge base

WARNING: This strategy is for internal use only. It is not possible to use Assistants from a "bring your own" OpenAI account.

In an LLM prompts file, a knowledge base is configured by adding a knowledge key:

prompts:
  - id: faq
    text: |
      system: use the given knowledge base to answer the users question.
      [!kb]
      user: {{ question }}
knowledge:
  - id: "my_knowledge_base"
    label: "My Knowledge Base"
    managed_openai_assistant: {}

This can then be used in bubblescript as follows:

dialog main do
  ask "What is the capital of France?"
  answer = LLM.complete(@prompts.faq, kb=@knowledge.my_knowledge_base, question=answer.text)
  say answer.text
end

In this case the AI section of the studio will have a 'knowledge bases' section, where you can upload files to the knowledge base.

Retrieval strategies

There are several strategies to use a knowledge base.

Managed OpenAI Vector store (Responses API)

The managed_openai_responses strategy automatically creates and manages an OpenAI Vector store for your knowledge base. Files uploaded to the bot's file system are automatically synchronized with the vector store.

knowledge:
  - id: "my_knowledge_base"
    label: "My Knowledge Base"
    managed_openai_responses: {}

This strategy will: - Automatically create an OpenAI Vector store for your bot - Synchronize files from the bot's file system to the vector store - Use the OpenAI Responses API with the file_search tool to search the vector store and incorporate the results in the LLM response.

It is possible to configure the number of results to return with the max_num_results field:

knowledge:
  - id: "my_knowledge_base"
    label: "My Knowledge Base"
    managed_openai_responses:
      max_num_results: 5

This example will retrieve up to 5 results from the vector store, instead of the default of 3.

Managed OpenAI Assistant (to be deprecated)

The managed_openai_assistant strategy automatically creates and manages an OpenAI Assistant for your knowledge base. Files uploaded to the bot's file system are automatically synchronized with the assistant's vector store.

WARNING: This strategy is deprecated and will be removed in the future. From release 2.46, newly created bots will use the managed_openai_responses strategy instead.

knowledge:
  - id: "my_knowledge_base"
    label: "My Knowledge Base"
    managed_openai_assistant:
      provider: "openai"

This strategy will: - Automatically create an OpenAI Assistant for your bot - Create a vector store attached to the assistant - Synchronize files from the bot's file system to the vector store - Use the assistant's file search capabilities to find relevant documents

The provider field should be set to openai or microsoft_openai and must match the provider of the LLM prompt that uses this knowledge base.

Files are automatically synchronized when: - Files are uploaded to the bot's file system - Files are renamed or moved - Files are deleted

The assistant and vector store are created automatically when first needed, and are named based on your bot ID and knowledge base ID.

Scripts collection

The scripts_collection strategy uses a collection of scripts as a knowledge base:

knowledge:
  - id: "my_knowledge_base"
    label: "My Knowledge Base"
    scripts_collection:
      collection: "kb/"

This will read all the scripts in the kb/ directory and use them as a knowledge base, by directly including the content of the scripts in the prompt, on the place where the [!kb] tag is set in the prompt definition.

Google Vertex AI RAG

The google_vertex_rag strategy uses a Vertex AI RAG Engine corpus as knowledge base. When the LLM runs, Gemini retrieves from the corpus during generation, instead of the retrieved text being inserted into the prompt.

Contrary to the other strategies, this strategy is not configured in a prompts file. Create these knowledge bases in the 'AI/Knowledge Bases' section of the studio, where you also give them their label and description. In bubblescript they are used like any other knowledge base, as @knowledge.my_vertex_kb.

Vertex AI knowledge bases are unversioned: create, update and delete apply immediately to both the draft and the published bot (there is no separate publish step for them).

The corpus is created automatically when first needed (on first file upload or publish). Its Vertex resource name is stored with the knowledge base and reused for later ingest, delete and completions. Files uploaded to the knowledge base are automatically synchronized into the corpus, just like for the managed OpenAI strategies.

Cloning a bot copies its Vertex AI knowledge bases, and a fresh corpus is created for the clone. Because these knowledge bases are not part of a prompts file, they are not included in a bot export.

Internal retrieval strategies

WARNING: These strategies are for internal use only. It is not possible to use Assistants from a "bring your own" OpenAI account.

OpenAI Responses API (hardcoded)

The fixed_openai_responses strategy is used to connect to an OpenAI Vector store by hardcoding the vector store ID.

knowledge:
  - id: "my_knowledge_base"
    label: "My Knowledge Base"
    fixed_openai_responses:
      vector_store_id: "vs_1234567890"
      max_num_results: 1

OpenAI Assistant through integration

The external_openai_assistant strategy is used to connect to an OpenAI Assistant by configuring the assistant ID in an integration secret.

knowledge:
  - id: "my_knowledge_base"
    label: "My Knowledge Base"
    external_openai_assistant:
      assistant_integration_alias: "my_assistant"
      # optional reference to alias where the project ID is stored
      project_integration_alias: "my_project"

And then in the integrations file, add a new entry like this:

- provider: secret
  alias: my_assistant
  context: bot
  description: Assistant ID for 'My Knowledge Base'
- provider: secret
  alias: my_project
  context: bot
  description: Project ID for 'My Knowledge Base'

No automatic synchronization of files is done from the platform to the assistant. With this retrieval strategy, you need to manually add files to the assistant on the OpenAI platform.

OpenAI Assistant (hardcoded)

The "fixed" OpenAI Assistant strategy. This strategy is configured using a hardcoded OpenAI Assistant ID to find the most relevant documents to include in the prompt.

knowledge:
  - id: "my_knowledge_base"
    label: "My Knowledge Base"
    fixed_openai_assistant:
      provider: openai
      assistant_id: "asst_1234567890"

To use this strategy, you need to provide the assistant_id of an OpenAI Assistant. For the openai provider, you can create an assistant here. For the microsoft_openai provider, you can create an assistant here. The provider should be set to openai or microsoft_openai; it decides which API endpoint to use. The provider of the knowledge base must match the provider of the LLM prompt that is using the knowledge base.

No automatic synchronization of files is done from the platform to the assistant. With this retrieval strategy, you need to manually add files to the assistant on the OpenAI platform.