Building data transformation workflows should be easy!
This section shows you how to create, manage and build a project workflow
Create a project
To create a project, you need to click on the "Create project" at the sidebar on the project list page
Fill in the project field to create a project which requires the name of the project and the cluster to use.
There is a need for you to have a cluster to use for your project so it is important to create or have existing clusters to use for the project creation.
Building a workflow for a project
Once you have created a project, it redirects you to the project workflow where you can perform data integration and transformation on your data. Our project workflow enables you to be able to ingest data from different databases, and perform different transformation processes like filtering, sorting, replacing, removing duplicates from datasets, and joining data in different datasets.
When using our workflow, there are parameters to understand and use to run your workflow process which include:
For you to build a workflow, there is n need for you to have sufficient data to perform the transformation process, you need to input data into the workflow. under the "Data" we have the "Input data field where you can ingest your data from different data connections and databases.
It is to be able to convert your data from different formats to well-designed and readable formats. Under this section, we have different transform processes which are broken into sections and sub-section. Under each section of the transform, there are sub-sections for each transform process. Under this section, we have Wrangle, Column operators, Filter and split and Group and Pivot transformation process.
Each of these transformation features is sub-divided into:
Filter and Split
Group and Pivot
Wrangle: Under the wrangle transform process, we have the Search/Replace dialog box and Drop duplicate dialog box. With these features, you will be able to clean up any messy datasets from your database. To use the "wrangle feature" click on the "Wrangle" under the transform feature and select the dialog box to use for your transformation
Column Operators: Under this column operator drop-down arrow, we have the Join, Sort, and Column-Rename you can use to transform your data.
-Filter and Split: Under the filter and split drop-down arrow, we have the Random split, Limit, Split by filter, Filter by value and Filter by SQL.
Group and Pivot: The group and pivot transform tools are processing tools that enable you to query, organize and summarize data between tables or databases. Under the Group and pivot transform drop-down arrow, we have the "Pivot, Unpivot and Group by" that can be used to perform your data transformation process.
This is part of the workflow in that you store or save the result of the transformation in a cloud database. Under this section, we have several outputs (database) that can enable storing data for further analysis. Click on the output dropdown to select a cloud database to save data. Voyance have several outputs like MySQL*, Redshift, Postgres, MongoDB, MariaDB, Microsoft SQL server, Azure SQL**. you can select any of the databases to save data.
Manage your Projects
This section describes how to manage the workflow of your project including displaying, editing and deleting.
Click on "Project" on the navigation bar to display your project page list.
Edit a project
To edit a project you have created, click on the "Edit icon " under the action tab on the display project list
Delete a project
To delete a project from the display list, click o the "Delete icon under the action tab to delete a project.
Updated 6 months ago