Concat

Concat files from Kestra’s internal storage.

yaml
type: "io.kestra.plugin.core.storage.Concat"

Examples

Concat 2 files with a custom separator.

yaml
id: "concat"
type: "io.kestra.plugin.core.storage.Concat"
files: 
  - "kestra://long/url/file1.txt"
  - "kestra://long/url/file2.txt"
separator: "\n"

Concat files generated by an each task.

yaml
tasks:
  - id: each
    type: io.kestra.plugin.core.flow.ForEach
    tasks:
      - id: start_api_call
        type: io.kestra.plugin.scripts.shell.Commands
        commands:
          - echo {{ taskrun.value }} > {{ temp.generated }}
        files:
          - generated
    values: '["value1", "value2", "value3"]'

  - id: concat
    type: io.kestra.plugin.core.storage.Concat
    files:
      - "{{ outputs.start_api_call.value1.files.generated }}"
      - "{{ outputs.start_api_call.value2.files.generated }}"
      - "{{ outputs.start_api_call.value3.files.generated }}"

Concat a dynamic number of files.

yaml
tasks:
  - id: echo
    type: io.kestra.plugin.scripts.shell.Commands
    commands:
      - echo "Hello John" > {{ outputDirs.output }}/1.txt
      - echo "Hello Jane" > {{ outputDirs.output }}/2.txt
      - echo "Hello Doe" > {{ outputDirs.output }}/3.txt
    outputDirs:
      - output
  - id: concat
    type: io.kestra.plugin.core.storage.Concat
    files: "{{ outputs.echo.files | jq('.[]') }}"

Properties

files *object

List of files to be concatenated

Must be kestra:// storage URIs; it can be a list of strings or a JSON string.

Pebble expression referencing an Internal Storage URI e.g. {{ outputs.mytask.uri }}.

extension string

Default .tmp

The extension of the created file — the default is .tmp

separator string

The separator to used between files — the default is no separator

Outputs

uri string

Format uri

The concatenated file URI

​Concat

Concat