Ignite Discussions : Ask Questions, Find Answers, Share Expertise about Sparkflows
How to achieve this?
In Sparkflows, we can use the ‘Row Filter’ processor to apply filter based on multiple conditions joined using ‘and’ and ‘or’ operators.
To use the ‘Row Filter’ Processor:
- Enter a ‘Conditional Expression’ consisting of multiple conditions joined together. An
example would be ‘salary > 20000 and dept_no = 10’. It would filter rows based on
the expression and send them to the outgoing DataFrame.
For more information read the Sparkflows Documentation here: