Ignite Discussions : Ask Questions, Find Answers, Share Expertise about Sparkflows
Please help me out.
To write conditions in the GE Decision node, follow these steps:
The GE Decision node requires an input DataFrame from a CSV file created from GE Results. This is typically done using the Create CSV from the GE Results node.
The input DataFrame should have four columns: expectation_type, column, success, and unexpected_percent.
To check values in the GE Decision node, specify any values at a time: column_name, expectation_type, and unexpected_percent.
For example, let's assume we have a dataset with the following CSV values:
To write a check condition, you can specify the values as follows:
Using the && (AND) or || (OR) operators, you can chain multiple conditions. For example:
ccnumber,expect_column_value_lengths_to_equal,false && expect_column_values_to_be_null,email,unexpected_percent > 80 || expect_column_values_to_be_null,id,false ||expect_column_value_lengths_to_be_between,postal,unexpected_percent = 0
Use the correct column names, expectation types, and logical operators per your specific requirements.