top of page

Spark Performance


Name

Description

Cache Data Frame

Caches the DataFrame with the provided StorageLevel.

Print Spark Configuration

Print the all spark configuration used in workflow.

Unpersist DataFrame

Unpersists the output DataFrames of the given Nodes.


Data Partition


Name

Description

Coalesce

This node coalesces the DataFrame into specified number of Partitions.

Number Of Partitions

This node will get the number partitions in input DataFrame.

Repartition

This node repartitions incoming DataFrame into a specified number of partitions.


Utilities


Name

Description

EmailNotification

This node sends notification to given email address with given content.

ExecuteRedshiftStatement

This node execute the Redshift statement.


bottom of page