---
agentTools:
  projectIndex: https://docs.voyancehq.com/llms.txt
---

# Clusters

A cluster is a set of computation applications and configurations that allow you to run data transformation workflows such as our ETL pipelines batch processing and machine learning.

This section focus on the creation of clusters with the different cluster mode, with different configuration and management tools that differs during creation. Learn how to create and manage your clusters

Getting started

**Create clusters**
**Manage clusters**

\###Create clusters

To create a cluster, click on the "Cluster" tab at the top of the navigation bar, and click on "Create cluster" at the sidebar.

[block:image]
{
  "images": [
    {
      "image": [
        "https://files.readme.io/22e3f81-Screenshot_2022-09-01_at_16.46.04.png",
        "Screenshot 2022-09-01 at 16.46.04.png",
        2870,
        1518,
        "#ebebf0"
      ]
    }
  ]
}
[/block]

it redirects you to the create cluster page where you will have to name and configure the cluster you intend to create for your workflow. There are some configuration options which will be discussed in detail.

[block:image]
{
  "images": [
    {
      "image": [
        "https://files.readme.io/8038eff-Screenshot_2022-05-10_at_14.46.19.png",
        "Screenshot 2022-05-10 at 14.46.19.png",
        2880,
        1641,
        "#eaebf1"
      ]
    }
  ]
}
[/block]

After filling in all the information needed for the cluster, then click on the "**Create cluster**" button at the bottom of the dashboard to submit. Under the cluster creation, there are different configuration parameters

**Cluster name:** Enter a name for the cluster you want to create in the cluster name field.

**Cluster mode:** Specify the cluster mode you tend to use for your configuration. The cluster mode you chose will determine the other configuration field that will appear. To select a cluster mode, click on the drop-down and select the cluster mode you want to use either "**Light job computes, Jobs computes, and All-purpose computes**" depending on what is best for your data and workflow.

[block:image]
{
  "images": [
    {
      "image": [
        "https://files.readme.io/f9be9e5-Screenshot_2022-05-10_at_16.20.06.png",
        "Screenshot 2022-05-10 at 16.20.06.png",
        2880,
        1642,
        "#e7e9f0"
      ]
    }
  ]
}
[/block]

**Autopilot Options**
When you are creating a cluster using our data platform, you can use the "**Autopilot option**" to reduce the operational cost of managing clusters and optimize clusters for production and yield higher workload availability.
The autopilot options for your clusters are the "**Enable Autoscaling option and the Manual Scaling option"**.

**Enable and Configure Autoscaling**
To enable our data platform to help you to resize your cluster automatically, you need to enable autoscaling for the cluster and provide the minimum and maximum numbers of workers.

**Enable Autoscaling**

* **Job Computes**: When you choose the "**Job computes cluster mode**", you can either enable autoscaling or manually determine the termination time.  On the create cluster page, select the **"Enable autoscaling"** checkbox in the autopilot option to activate autoscaling.

**Terminate After**: When you choose to terminate the job manually, click on the "**Terminate after**  checkbox and manually choose or select the minutes of inactivity.

[block:image]
{
  "images": [
    {
      "image": [
        "https://files.readme.io/7a78ee0-Screenshot_2022-05-10_at_16.59.05.png",
        "Screenshot 2022-05-10 at 16.59.05.png",
        2880,
        1640,
        "#eaebf1"
      ]
    }
  ]
}
[/block]

**All-Purpose Computes**  When you select the "**All-purpose computes cluster mode**, you have the option to select any of the autopilot options.  On the create cluster page, select the "**Enable autoscaling"** checkbox in the autopilot option to activate autoscaling.

[block:image]
{
  "images": [
    {
      "image": [
        "https://files.readme.io/9af8669-Screenshot_2022-05-10_at_16.56.00.png",
        "Screenshot 2022-05-10 at 16.56.00.png",
        2874,
        1632,
        "#ebecf1"
      ]
    }
  ]
}
[/block]

[block:callout]
{
  "type": "info",
  "title": "Note",
  "body": "It is until you enable the autoscaling option when creating your cluster that you will be able to provide a minimum and maximum worker for your cluster. When you choose the auto termination option alone, you will only be able to choose the number of workers."
}
[/block]

[block:callout]
{
  "type": "info",
  "title": "Tips",
  "body": "Autoscaling is not available for Light Job computes but it is available for Job computes and All-purpose computes"
}
[/block]

**Workers Type:**
To select the workers type, click on the "Workers type" and select the worker type you want to use for cluster from the drop-down list.  you have the option of providing a minimum and maximum number of worker for your cluster. The Job computes and All-purpose computes cluster modes works with workers type.

To select a minmum and maximum worker for your cluster, click on the drop-down arrows to select the best number of workers for your cluster.

[block:image]
{
  "images": [
    {
      "image": [
        "https://files.readme.io/6d5da2c-Screenshot_2022-05-10_at_19.40.38.png",
        "Screenshot 2022-05-10 at 19.40.38.png",
        2880,
        1642,
        "#e6e8f0"
      ]
    }
  ]
}
[/block]

**Supervisor Type**
To select your supervisor type, click on the "Supervisor type" and select a type from the drop-down list for your cluster. The Job computes and All-purpose computes cluster modes works with supervisor type.

[block:image]
{
  "images": [
    {
      "image": [
        "https://files.readme.io/dd61c74-Screenshot_2022-05-10_at_19.45.17.png",
        "Screenshot 2022-05-10 at 19.45.17.png",
        2880,
        1628,
        "#ecedf2"
      ]
    }
  ]
}
[/block]

[block:callout]
{
  "type": "info",
  "title": "Tips",
  "body": "Light Job computes does not support worker type and supervisor type."
}
[/block]

**Node Type**
When you are creating a cluster for your workflow and you choose the Light Job computes, the "Node Types" appears for your configuration.  To select a node type, click on the "Node Type" drop-down list to select a node for your cluster.

[block:image]
{
  "images": [
    {
      "image": [
        "https://files.readme.io/1ff58d2-Screenshot_2022-05-11_at_09.05.52.png",
        "Screenshot 2022-05-11 at 09.05.52.png",
        2880,
        1628,
        "#eceef3"
      ]
    }
  ]
}
[/block]

[block:callout]
{
  "type": "info",
  "title": "Tips",
  "body": "The Node Type does not support the Job computes and All-purpose computes cluster mode"
}
[/block]

[block:api-header]
{
  "title": "Manage Cluster"
}
[/block]

This section describes how to manage the created cluster including displaying, cloning, restarting, terminating and deleting and monitoring the performance of the clusters.

**Display cluster**
Click on "cluster" at the navigation bar to display your clusters.

[block:image]
{
  "images": [
    {
      "image": [
        "https://files.readme.io/6990033-Screenshot_2022-05-11_at_09.46.38.png",
        "Screenshot 2022-05-11 at 09.46.38.png",
        2880,
        1534,
        "#ebecf2"
      ]
    }
  ]
}
[/block]

With this display, you can view the details of every cluster created which allows you to manage the clusters. The following parameter of the display page includes:

Cluster name
State of cluster
Nodes
Workers
Supervisor
Autoscaling
Created on
Action

**Cloning a cluster**
You can clone a cluster from the display page of a cluster. Click on the " Clone icon" under the action tab.

[block:image]
{
  "images": [
    {
      "image": [
        "https://files.readme.io/006089e-Screenshot_2022-05-11_at_11.00.10.png",
        "Screenshot 2022-05-11 at 11.00.10.png",
        2880,
        1642,
        "#eaebf1"
      ]
    }
  ]
}
[/block]

you can clone a cluster while creating a cluster from the cluster detail page.

[block:image]
{
  "images": [
    {
      "image": [
        "https://files.readme.io/8bbd1bf-Screenshot_2022-05-11_at_11.03.39.png",
        "Screenshot 2022-05-11 at 11.03.39.png",
        2880,
        1646,
        "#ebecf1"
      ]
    }
  ]
}
[/block]

**Restart a cluster**
You can restart a previously terminated cluster from the cluster display page which allows you to resume a previously terminated cluster.

you can restart a cluster from the cluster display list or the cluster detail page.

[block:image]
{
  "images": [
    {
      "image": [
        "https://files.readme.io/f4d18f2-Screenshot_2022-05-11_at_11.11.49.png",
        "Screenshot 2022-05-11 at 11.11.49.png",
        2880,
        1636,
        "#ebecf2"
      ]
    }
  ]
}
[/block]

From the cluster detail page

[block:image]
{
  "images": [
    {
      "image": [
        "https://files.readme.io/e50367a-Screenshot_2022-05-11_at_11.11.49.png",
        "Screenshot 2022-05-11 at 11.11.49.png",
        2880,
        1636,
        "#ebecf2"
      ]
    }
  ]
}
[/block]

**Terminate/Stop a cluster**
You can manually terminate a cluster or configure the cluster to automatically terminate after a specified period of inactivity. A terminated cluster cannot run a job(s).

**Manual termination**
you can manually terminate a cluster from the display page list.

[block:image]
{
  "images": [
    {
      "image": [
        "https://files.readme.io/21b76a6-Screenshot_2022-05-11_at_11.23.31.png",
        "Screenshot 2022-05-11 at 11.23.31.png",
        2880,
        1640,
        "#eaebf1"
      ]
    }
  ]
}
[/block]

You can also manually terminate a cluster from the cluster detail page

[block:image]
{
  "images": [
    {
      "image": [
        "https://files.readme.io/7d4251e-Screenshot_2022-05-11_at_11.24.44.png",
        "Screenshot 2022-05-11 at 11.24.44.png",
        2880,
        1642,
        "#eaebf0"
      ]
    }
  ]
}
[/block]

**Automatic termination**
You have the option to auto terminate a cluster after a certain period of inactivity. When creating a cluster, under the autopilot option, you can set an inactivity period in minutes after which you want the cluster to terminate.
you can configure an automatic termination in the auto termination field under the autopilot option on the cluster creation page.

Click on the **Automatic termination checkbox** and set a time in minutes for inactivity.

[block:image]
{
  "images": [
    {
      "image": [
        "https://files.readme.io/fc0b5d1-Screenshot_2022-05-11_at_11.46.53.png",
        "Screenshot 2022-05-11 at 11.46.53.png",
        2880,
        1646,
        "#eaebf1"
      ]
    }
  ]
}
[/block]

You can opt-out of the auto termination by clicking on the auto termination checkbox or by specifying the inactivity time period to be 0.

Delete a cluster
when you delete a cluster, it permanently terminates and removes the configuration of the cluster.

[block:callout]
{
  "type": "danger",
  "title": "Warning",
  "body": "Once you delete a cluster, you cannot undo the action"
}
[/block]

you can delete a cluster from the cluster display list or cluster detail page.

[block:image]
{
  "images": [
    {
      "image": [
        "https://files.readme.io/da34d8a-Screenshot_2022-05-11_at_12.47.56.png",
        "Screenshot 2022-05-11 at 12.47.56.png",
        2880,
        1632,
        "#ebedf2"
      ]
    }
  ]
}
[/block]

from the cluster detail page

[block:image]
{
  "images": [
    {
      "image": [
        "https://files.readme.io/42c1ef3-Screenshot_2022-05-11_at_12.45.21.png",
        "Screenshot 2022-05-11 at 12.45.21.png",
        2880,
        1634,
        "#ebecf1"
      ]
    }
  ]
}
[/block]