Esql

Query Elasticsearch using ES|QL.

yaml
type: "io.kestra.plugin.elasticsearch.Esql"

Examples

Load data in bulk to Elasticsearch and query it using ES|QL.

yaml
id: bulk_load_and_query
namespace: company.team

tasks:
  - id: extract
    type: io.kestra.plugin.core.http.Download
    uri: https://huggingface.co/datasets/kestra/datasets/resolve/main/jsonl/books.jsonl

  - id: load
    type: io.kestra.plugin.elasticsearch.Bulk
    from: "{{ outputs.extract.uri }}"

  - id: sleep
    type: io.kestra.plugin.core.flow.Sleep
    duration: PT5S
    description: Pause needed after load before we can query

  - id: query
    type: io.kestra.plugin.elasticsearch.Esql
    fetchType: STORE
    query: |
      FROM books
        | KEEP author, name, page_count, release_date
        | SORT page_count DESC
        | LIMIT 5

pluginDefaults:
  - type: io.kestra.plugin.elasticsearch
    values:
      connection:
        headers:
          - "Authorization: ApiKey yourEncodedApiKey"
        hosts:
          - https://yourCluster.us-central1.gcp.cloud.es.io:443

Properties

connection *ElasticsearchConnection

The connection properties.

query *string

The ElasticSearch value.

Can be a JSON string. In this case, the contentType will be used or a raw Map.

fetchType string

Default FETCH

Possible Values

STOREFETCHFETCH_ONENONE

The way you want to store the data.

FETCH_ONE output the first row, FETCH output all the rows, STORE store all rows in a file, NONE do nothing.

filter object

Query filter.

Specify a DSL query in the filter parameter to filter the set of documents that an ES|QL query runs on.

routing string

Controls the shard routing of the request.

Using this value to hash the shard and not the id.

Outputs

row object

Map containing the first row of fetched data.

Only populated if using fetchType=FETCH_ONE.

rows array

SubType object

List containing the fetched data.

Only populated if using fetchType=FETCH.

size integer

The number of fetched rows.

total integer

The total number of rows fetched without pagination.

uri string

Format uri

The URI of the data stored in Kestra's internal storage.

Only populated if using fetchType=STORE.

Definitions

io.kestra.plugin.elasticsearch.ElasticsearchConnection

hosts *array

SubType string

Min items 1

List of HTTP ElasticSearch servers.

Must be an URI like https://elasticsearch.com: 9200 with scheme and port.

basicAuth ElasticsearchConnection-BasicAuth

Basic auth configuration.

headers array

SubType string

List of HTTP headers to be send on every request.

Must be a string with key value separated with : , ex: Authorization: Token XYZ.

pathPrefix string

Sets the path's prefix for every request used by the HTTP client.

For example, if this is set to /my/path, then any client request will become /my/path/ + endpoint. In essence, every request's endpoint is prefixed by this pathPrefix. The path prefix is useful for when ElasticSearch is behind a proxy that provides a base path or a proxy that requires all paths to start with '/'; it is not intended for other purposes and it should not be supplied in other scenarios.

strictDeprecationMode booleanstring

Whether the REST client should return any response containing at least one warning header as a failure.

trustAllSsl booleanstring

Trust all SSL CA certificates.

Use this if the server is using a self signed SSL certificate.

io.kestra.plugin.elasticsearch.ElasticsearchConnection-BasicAuth

password string

Basic auth password.

username string

Basic auth username.

​Esql

Esql