Create

Create a Google Cloud Dataproc cluster.

yaml
type: "io.kestra.plugin.gcp.dataproc.clusters.Create"

Examples

Creates a cluster in Google Cloud Dataproc.

yaml
id: gcp_dataproc_cluster_create
namespace: company.team

tasks:
  - id: cluster_create
    type: io.kestra.plugin.gcp.dataproc.clusters.Create
    clusterName: YOUR_CLUSTER_NAME
    region: YOUR_REGION
    zone: YOUR_ZONE
    masterMachineType: n1-standard-2
    workerMachineType: n1-standard-2
    workers: 2
    bucket: YOUR_BUCKET_NAME

Creates a cluster in Google Cloud Dataproc with specific disk size.

yaml
id: gcp_dataproc_cluster_create
namespace: company.team

tasks:
  - id: create_cluster_with_certain_disk_size
    type: io.kestra.plugin.gcp.dataproc.clusters.Create
    clusterName: YOUR_CLUSTER_NAME
    region: YOUR_REGION
    zone: YOUR_ZONE
    masterMachineType: n1-standard-2
    masterDiskSizeGB: 500
    workerMachineType: n1-standard-2
    workerDiskSizeGB: 200
    workers: 2
    bucket: YOUR_BUCKET_NAM

Properties

clusterName *string

Min length 1

The cluster name.

region *string

Min length 1

The region.

bucket string

The GCS bucket name.

imageVersion string

The Dataproc image URI.

The Compute Engine image resource used for cluster instances.

impersonatedServiceAccount string

The GCP service account to impersonate.

masterDiskSizeGB integerstring

The disk size in GB for each master node.

masterMachineType string

The master machine type.

projectId string

The GCP project ID.

scopes array

SubType string

Default ["https://www.googleapis.com/auth/cloud-platform"]

The GCP scopes to be used.

serviceAccount string

The GCP service account.

workerDiskSizeGB integerstring

The disk size in GB for each worker node.

workerMachineType string

The worker machine type.

workers integerstring

The number of workers.

zone string

The zone.

Outputs

clusterName string

The cluster name.

created boolean

Default false

Whether cluster was created successfully.

​Create

Create