Churn Prediction using Sparkflows

In this solution, we create a Random Forest Model to predict churn and evaluate the results.

​

Dataset

​

The dataset is artificial Churn Data based on claims, similar to real world. It is taken from the following location. 

• https://www.sgi.com/tech/mlc/db/

• https://www.sgi.com/tech/mlc/db/churn.all

• https://www.sgi.com/tech/mlc/db/churn.name

Sample Data

​

KS, 128, 415, 382-4657, no, yes, 25, 265.1, 110, 45.07, 197.4, 99, 16.78, 244.7, 91, 11.01, 10, 3, 2.7, 1, False.

OH, 107, 415, 371-7191, no, yes, 26, 161.6, 123, 27.47, 195.5, 103, 16.62, 254.4, 103, 11.45, 13.7, 3, 3.7, 1, False.

NJ, 137, 415, 358-1921, no, no, 0, 243.4, 114, 41.38, 121.2, 110, 10.3, 162.6, 104, 7.32, 12.2, 5, 3.29, 0, False.

OH, 84, 408, 375-9999, yes, no, 0, 299.4, 71, 50.9, 61.9, 88, 5.26, 196.9, 89, 8.86, 6.6, 7, 1.78, 2, False.

OK, 75, 415, 330-6626, yes, no, 0, 166.7, 113, 28.34, 148.3, 122, 12.61, 186.9, 121, 8.41, 10.1, 3, 2.73, 3, False.

Dataset Fields

State: discrete.
Account length: continuous.
Area code: continuous.
Phone number: discrete.
International plan: discrete.
Voice mail plan: discrete.
Numbervmail messages: continuous.
Total day minutes: continuous.
Total day calls: continuous.
Total day charge: continuous.
Total eve minutes: continuous.
Total eve calls: continuous.
Total eve charge: continuous.
Total night minutes: continuous.
Total night calls: continuous.
Total night charge: continuous.
Totalintl minutes: continuous.
Totalintl calls: continuous.
Totalintl charge: continuous.
Number customer service calls: continuous.

​

​

​

Workflow

Below is the workflow we use for creating the model for Churn Prediction.

​

The workflow performs the following steps:​

​

• Reads in the dataset from a tab separated file

• Applies StringIndexer on the field intl_plan

• Applies VectorAssembler on the fields we want to model on

• Splits the dataset into (.8, .2)

• Performs Random Forest Classification

• Performs prediction using the model generated on the remaining 20% dataset

• Finally evaluates the prediction results

String Indexer

VectorAssembler

​

In the VectorAssembler, we select the fields we want to include in the model. Only the numeric fields are displayed as VectorAssembler supports only the numeric fields.

Split

​

Here we split the dataset into training and test datasets. We split it into (.8, .2)

RandomForestClassifier

​

Here we use a RandomForestClassifier for predicting churn. We use 20 trees.

Predict

​

Here we predict using the model on the test dataset.

BinaryClassificationEvaluator

​

Here we evaluate the quality of our results.

Executing the Workflow

​

Next, we execute the workflow. We come up with the below model. It is a forest with 20 trees.

From the evaluator we get the following results:

Workflow JSON

​

The workflow consists of the below JSON. The workflow can be run interactively from the Sparkflows UI, or it can be easily schedule with spark-submit with any scheduler.