ChatCompletion​Chat​Completion

Create a Retrieval Augmented Generation (RAG) pipeline

yaml
type: "io.kestra.plugin.ai.rag.ChatCompletion"

Chat with your data using Retrieval Augmented Generation (RAG). This flow will index documents and use the RAG Chat task to interact with your data using natural language prompts. The flow contrasts prompts to LLM with and without RAG. The Chat with RAG retrieves embeddings stored in the KV Store and provides a response grounded in data rather than hallucinating. WARNING: the Kestra KV embedding store is for quick prototyping only, as it stores the embedding vectors in Kestra's KV store and loads them all into memory.

yaml
id: rag
namespace: company.ai

tasks:
  - id: ingest
    type: io.kestra.plugin.ai.rag.IngestDocument
    provider:
      type: io.kestra.plugin.ai.provider.GoogleGemini
      modelName: gemini-embedding-exp-03-07
      apiKey: "{{ kv('GEMINI_API_KEY') }}"
    embeddings:
      type: io.kestra.plugin.ai.embeddings.KestraKVStore
    drop: true
    fromExternalURLs:
      - https://raw.githubusercontent.com/kestra-io/docs/refs/heads/main/content/blogs/release-0-24.md

  - id: parallel
    type: io.kestra.plugin.core.flow.Parallel
    tasks:
      - id: chat_without_rag
        type: io.kestra.plugin.ai.completion.ChatCompletion
        provider:
          type: io.kestra.plugin.ai.provider.GoogleGemini
        messages:
          - type: USER
            content: Which features were released in Kestra 0.24?

      - id: chat_with_rag
        type: io.kestra.plugin.ai.rag.ChatCompletion
        chatProvider:
          type: io.kestra.plugin.ai.provider.GoogleGemini
        embeddingProvider:
          type: io.kestra.plugin.ai.provider.GoogleGemini
          modelName: gemini-embedding-exp-03-07
        embeddings:
          type: io.kestra.plugin.ai.embeddings.KestraKVStore
        systemMessage: You are a helpful assistant that can answer questions about Kestra.
        prompt: Which features were released in Kestra 0.24?

pluginDefaults:
  - type: io.kestra.plugin.ai.provider.GoogleGemini
    values:
      apiKey: "{{ kv('GEMINI_API_KEY') }}"
      modelName: gemini-2.5-flash

RAG chat with a web search content retriever (answers grounded in search results)

yaml
id: rag_with_websearch_content_retriever
namespace: company.ai

tasks:
  - id: chat_with_rag_and_websearch_content_retriever
    type: io.kestra.plugin.ai.rag.ChatCompletion
    chatProvider:
      type: io.kestra.plugin.ai.provider.GoogleGemini
      modelName: gemini-2.5-flash
      apiKey: "{{ kv('GEMINI_API_KEY') }}"
    contentRetrievers:
      - type: io.kestra.plugin.ai.retriever.TavilyWebSearch
        apiKey: "{{ kv('TAVILY_API_KEY') }}"
    systemMessage: You are a helpful assistant that can answer questions about Kestra.
    prompt: What is the latest release of Kestra?

Store chat memory as a Kestra KV pair

yaml
id: chat_with_memory
namespace: company.ai

inputs:
  - id: first
    type: STRING
    defaults: Hello, my name is John and I'm from Paris

  - id: second
    type: STRING
    defaults: What's my name and where do I live?

tasks:
  - id: first
    type: io.kestra.plugin.ai.rag.ChatCompletion
    chatProvider:
      type: io.kestra.plugin.ai.provider.GoogleGemini
    embeddingProvider:
      type: io.kestra.plugin.ai.provider.GoogleGemini
      modelName: gemini-embedding-exp-03-07
    embeddings:
      type: io.kestra.plugin.ai.embeddings.KestraKVStore
    memory:
      type: io.kestra.plugin.ai.memory.KestraKVStore
      ttl: PT1M
    systemMessage: You are a helpful assistant, answer concisely
    prompt: "{{inputs.first}}"

  - id: second
    type: io.kestra.plugin.ai.rag.ChatCompletion
    chatProvider:
      type: io.kestra.plugin.ai.provider.GoogleGemini
    embeddingProvider:
      type: io.kestra.plugin.ai.provider.GoogleGemini
      modelName: gemini-embedding-exp-03-07
    embeddings:
      type: io.kestra.plugin.ai.embeddings.KestraKVStore
    memory:
      type: io.kestra.plugin.ai.memory.KestraKVStore
    systemMessage: You are a helpful assistant, answer concisely
    prompt: "{{inputs.second}}"

pluginDefaults:
  - type: io.kestra.plugin.ai.provider.GoogleGemini
    values:
      apiKey: "{{ kv('GEMINI_API_KEY') }}"
      modelName: gemini-2.5-flash

Classify recent Kestra releases into MINOR or PATCH using a JSON schema. Note: not all LLMs support structured outputs, or they may not support them when combined with tools like web search. This example uses Mistral, which supports structured output with content retrievers.

yaml
id: chat_with_structured_output
namespace: company.ai

tasks:
  - id: categorize_releases
    type: io.kestra.plugin.ai.rag.ChatCompletion
    chatProvider:
      type: io.kestra.plugin.ai.provider.MistralAI
      apiKey: "{{ kv('MISTRAL_API_KEY') }}"
      modelName: open-mistral-7b

    contentRetrievers:
      - type: io.kestra.plugin.ai.retriever.TavilyWebSearch
        apiKey: "{{ kv('TAVILY_API_KEY') }}"
        maxResults: 8

    chatConfiguration:
      responseFormat:
        type: JSON
        jsonSchema:
          type: object
          required: ["releases"]
          properties:
            releases:
              type: array
              minItems: 1
              items:
                type: object
                additionalProperties: false
                required: ["version", "date", "semver"]
                properties:
                  version:
                    type: string
                    description: "Release tag, e.g., 0.24.0"
                  date:
                    type: string
                    description: "Release date"
                  semver:
                    type: string
                    enum: ["MINOR", "PATCH"]
                  summary:
                    type: string
                    description: "Short plain-text summary (optional)"

    systemMessage: |
      You are a release analyst. Use the Tavily web retriever to find recent Kestra releases.
      Determine each release's SemVer category:
        - MINOR: new features, no major breaking changes (y in x.Y.z)
        - PATCH: bug fixes/patches only (z in x.y.Z)
      Return ONLY valid JSON matching the schema. No prose, no extra keys.

    prompt: |
      Find most recent Kestra releases (within the last ~6 months).
      Output their version, release date, semver category, and a one-line summary.
Properties

Chat model provider

User prompt

The user input for this run. May be templated from flow inputs.

Default {}

Chat configuration

Default { "maxResults": 3, "minScore": 0 }

Content retriever configuration

Additional content retrievers

Some content retrievers like WebSearch can also be used as tools, but using them as content retrievers will ensure that they are always called whereas tools are only used when the LLM decides to.

Embedding model provider

Optional. If not set, the embedding model is created from chatProvider. Ensure the chosen chat provider supports embeddings.

Embedding store

Optional when at least one entry is provided in contentRetrievers.

Chat memory

Stores conversation history and injects it into context on subsequent runs.

System message

Instruction that sets the assistant's role, tone, and constraints for this task.

Optional tools the LLM may call to augment its response

Possible Values
STOPLENGTHTOOL_EXECUTIONCONTENT_FILTEROTHER

Finish reason

Intermediate responses

LLM output for JSON response format

The result of the LLM completion for response format of type JSON, null otherwise.

SubType string

URIs of the generated files in Kestra's internal storage

Request duration in milliseconds

LLM output for TEXT response format

The result of the LLM completion for response format of type TEXT (default), null otherwise.

Token usage

Tool executions

The database name

The database server host

The database password

The database server port

The table to store embeddings in

The database user

Default false

Whether to use use an IVFFlat index

An IVFFlat index divides vectors into lists, and then searches a subset of those lists closest to the query vector. It has faster build times and uses less memory than HNSW but has lower query performance (in terms of speed-recall tradeoff).

The collection name

The host

The index name

The scheme (e.g., mongodb+srv)

Create the index

The database

SubType string

The metadata field names

The connection string options

The password

The username

API Key

Model name

API base URL

SubType string

MCP client command, as a list of command parts

SubType string

Environment variables

Default false

Log events

The database base URL

The collection name

The database server host

The database server port

Default embedding-index

The index name

Description of the flow if not already provided inside the flow itself

Use it only if you define the flow in the tool definition. The LLM needs a tool description to identify whether to call it. If the flow has a description, the tool will use it. Otherwise, the description property must be explicitly defined.

Flow ID of the flow that should be called

Default false

Whether the flow should inherit labels from this execution that triggered it

By default, labels are not inherited. If you set this option to true, the flow execution will inherit all labels from the agent's execution. Any labels passed by the LLM will override those defined here.

Input values that should be passed to flow's execution

Any inputs passed by the LLM will override those defined here.

Labels that should be added to the flow's execution

Any labels passed by the LLM will override those defined here.

Namespace of the flow that should be called

Revision of the flow that should be called

Format date-time

Schedule the flow execution at a later date

If the LLM sets a scheduleDate, it will override the one defined here.

Basic authorization password

Basic authorization username

URL of the MCP server

SubType string

Custom headers

Useful, for example, for adding authentication tokens via the Authorization header.

Default false

Log requests

Default false

Log responses

Format duration

Connection timeout duration

SubType

List of Kestra runnable tasks

API Key

Model name

Default https://api.deepseek.com/v1

API base URL

The API key

The cloud provider

The index

The cloud provider region

The namespace (default will be used if not provided)

Generated text completion

The result of the text completion

Possible Values
STOPLENGTHTOOL_EXECUTIONCONTENT_FILTEROTHER

Finish reason

Response identifier

Request duration in milliseconds

Token usage

Tool execution requests

JSON Schema (used when type = JSON)

Provide a JSON Schema describing the expected structure of the response. In Kestra flows, define the schema in YAML (it is still a JSON Schema object). Example (YAML):

text
responseFormat: 
    type: JSON
    jsonSchema: 
      type: object
      required: ["category", "priority"]
      properties: 
        category: 
          type: string
          enum: ["ACCOUNT", "BILLING", "TECHNICAL", "GENERAL"]
        priority: 
          type: string
          enum: ["LOW", "MEDIUM", "HIGH"]

Note: Provider support for strict schema enforcement varies. If unsupported, guide the model about the expected output structure via the prompt and validate downstream.

Schema description (optional)

Natural-language description of the schema to help the model produce the right fields. Example: "Classify a customer ticket into category and priority."

Default TEXT
Possible Values
TEXTJSON

Response format type

Specifies how the LLM should return output. Allowed values:

  • TEXT (default): free-form natural language.
  • JSON: structured output validated against a JSON Schema.

Container image

API version

SubType string

Volume binds

SubType string

MCP client command, as a list of command parts

Docker certificate path

Docker configuration

Docker context

Docker host

Whether Docker should verify TLS certificates

SubType string

Environment variables

Default false

Whether to log events

Container registry email

Container registry password

Container registry URL

Container registry username

API key

Custom search engine ID (cx)

Model endpoint

Model name

RapidAPI key for Judge0

You can obtain it from the RapidAPI website.

API Key

Model name

API base URL

API key

Custom search engine ID (cx)

Default 3

Maximum number of results

Log LLM requests

If true, prompts and configuration sent to the LLM will be logged at INFO level.

Log LLM responses

If true, raw responses from the LLM will be logged at INFO level.

Response format

Defines the expected output format. Default is plain text. Some providers allow requesting JSON or schema-constrained outputs, but support varies and may be incompatible with tool use. When using a JSON schema, the output will be returned under the key jsonOutput.

Seed

Optional random seed for reproducibility. Provide a positive integer (e.g., 42, 1234). Using the same seed with identical settings produces repeatable outputs.

Temperature

Controls randomness in generation. Typical range is 0.0–1.0. Lower values (e.g., 0.2) make outputs more focused and deterministic, while higher values (e.g., 0.7–1.0) increase creativity and variability.

Top-K

Limits sampling to the top K most likely tokens at each step. Typical values are between 20 and 100. Smaller values reduce randomness; larger values allow more diverse outputs.

Top-P (nucleus sampling)

Selects from the smallest set of tokens whose cumulative probability is ≤ topP. Typical values are 0.8–0.95. Lower values make the output more focused, higher values increase diversity.

The name of the index to store embeddings

Default 3

Maximum results to return from the embedding store

Default 0

Minimum similarity score (0-1 inclusive). Only results with score ≥ minScore are returned.

Tool request arguments

Tool execution request identifier

Tool name

API endpoint

The Azure OpenAI endpoint in the format: https://{resource}.openai.azure.com/

Model name

API Key

Client ID

Client secret

API version

Tenant ID

The API key

The collection name

The database server host

The database server port

Endpoint URL

Project location

Model name

Project ID

API Key

Model name

Default {{flow.id}}-embedding-store

The name of the KV pair to use

SSE URL of the MCP server

SubType string

Custom headers

Could be useful, for example, to add authentication tokens via the Authorization header.

Default false

Log requests

Default false

Log responses

Format duration

Connection timeout duration

API Key

Default 3

Maximum number of results to return

Redis host

The hostname of your Redis server (e.g., localhost or redis-server)

Default NEVER
Possible Values
NEVERBEFORE_TASKRUNAFTER_TASKRUN

Drop memory: never, before, or after the agent's task run

By default, the memory ID is the value of the system.correlationId label, meaning that the same memory will be used by all tasks of the flow and its subflows. If you want to remove the memory eagerly (before expiration), you can set drop: AFTER_TASKRUN to erase the memory after the taskrun. You can also set drop: BEFORE_TASKRUN to drop the memory before the taskrun.

Default {{ labels.system.correlationId }}

Memory ID - defaults to the value of the system.correlationId label. This means that a memory is valid for the entire flow execution including its subflows.

Default 10

Maximum number of messages to keep in memory. If memory is full, the oldest messages will be removed in a FIFO manner. The last system message is always kept.

Default 6379

Redis port

The port of your Redis server

Default PT1H
Format duration

Memory duration - defaults to 1h

Token

Milvus auth token. Required if authentication is enabled; omit for local deployments without auth.

Auto flush on delete

If true, flush after delete operations.

Auto flush on insert

If true, flush after insert operations. Setting it to false can improve throughput.

Collection name

Target collection. Created automatically if it does not exist. Default: "default".

Consistency level

Read/write consistency level. Common values include STRONG, BOUNDED, or EVENTUALLY (depends on client/version).

Database name

Logical database to use. If not provided, the default database is used.

Host

Milvus host name (used when uri is not set). Default: "localhost".

ID field name

Field name for document IDs. Default depends on collection schema.

Index type

Vector index type (e.g., IVF_FLAT, IVF_SQ8, HNSW). Depends on Milvus deployment and dataset.

Metadata field name

Field name for metadata. Default depends on collection schema.

Metric type

Similarity metric (e.g., L2, IP, COSINE). Should match the embedding provider’s expected metric.

Password

Required when authentication/TLS is enabled. See https://milvus.io/docs/authenticate.md

Port

Milvus port (used when uri is not set). Typical: 19530 (gRPC) or 9091 (HTTP). Default: 19530.

Retrieve embeddings on search

If true, return stored embeddings along with matches. Default: false.

Text field name

Field name for original text. Default depends on collection schema.

URI

Connection URI. Use either uri OR host/port (not both). Examples:

  • gRPC (typical): "milvus://host: 19530"
  • HTTP: "http://host: 9091"

Username

Required when authentication/TLS is enabled. See https://milvus.io/docs/authenticate.md

Vector field name

Field name for the embedding vector. Must match the index definition and embedding dimensionality.

API Key

Model name

Tavily API Key - you can obtain one from the Tavily website

Default NEVER
Possible Values
NEVERBEFORE_TASKRUNAFTER_TASKRUN

Drop memory: never, before, or after the agent's task run

By default, the memory ID is the value of the system.correlationId label, meaning that the same memory will be used by all tasks of the flow and its subflows. If you want to remove the memory eagerly (before expiration), you can set drop: AFTER_TASKRUN to erase the memory after the taskrun. You can also set drop: BEFORE_TASKRUN to drop the memory before the taskrun.

Default {{ labels.system.correlationId }}

Memory ID - defaults to the value of the system.correlationId label. This means that a memory is valid for the entire flow execution including its subflows.

Default 10

Maximum number of messages to keep in memory. If memory is full, the oldest messages will be removed in a FIFO manner. The last system message is always kept.

Default PT1H
Format duration

Memory duration - defaults to 1h

API key

Weaviate API key. Omit for local deployments without auth.

Host

Cluster host name without protocol, e.g., "abc123.weaviate.network".

Avoid duplicates

If true (default), a hash-based ID is derived from each text segment to prevent duplicates. If false, a random ID is used.

Possible Values
ONEQUORUMALL

Consistency level

Write consistency: ONE, QUORUM (default), or ALL.

gRPC port

Port for gRPC if enabled (e.g., 50051).

Metadata field name

Field used to store metadata. Defaults to "_metadata" if not set.

SubType string

Metadata keys

The list of metadata keys to store - if not provided, it will default to an empty list.

Object class

Weaviate class to store objects in (must start with an uppercase letter). Defaults to "Default" if not set.

Port

Optional port (e.g., 443 for https, 80 for http). Leave unset to use provider defaults.

Scheme

Cluster scheme: "https" (recommended) or "http".

Secure gRPC

Whether the gRPC connection is secured (TLS).

Use gRPC for batch inserts

If true, use gRPC for batch inserts. HTTP remains required for search operations.

SubType string
Min items 1

List of HTTP Elasticsearch servers

Must be a URI like https://example.com: 9200 with scheme and port

Basic authorization configuration

SubType string

List of HTTP headers to be sent with every request

Each item is a key: value string, e.g., Authorization: Token XYZ

Path prefix for all HTTP requests

If set to /my/path, each client request becomes /my/path/ + endpoint. Useful when Elasticsearch is behind a proxy providing a base path; do not use otherwise.

Treat responses with deprecation warnings as failures

Trust all SSL CA certificates

Use this if the server uses a self-signed SSL certificate

AWS Access Key ID

Model name

AWS Secret Access Key

Default COHERE
Possible Values
COHERETITAN

Amazon Bedrock Embedding Model Type