Ignite Discussions : Ask Questions, Find Answers, Share Expertise about Sparkflows
Can someone help me out with it?
To perform a data quality check on customer ID the following step needs to be followed:
- Create a PySpark workflow
- Load the input data using a Read processor.
- Insert and connect “ExpectColumnValueLengthstobeEqual” processor.
- Open and set the column name as Customer_ID and the value to 15. And Press Ok.
- Insert and Connect the Print N Rows processor to view the final output on the