Sparkflows is leading the effort to create the best in the world product for Self-Serve Data Prep & Analytics/Machine Learning. Being browser based end to end, scaling to petabytes of data and running seamlessly on the Cloud and on-premise makes it a must have product for any organization.


Come join us to change the whole definition of self-serve analytics! If you are looking to solve the challenges of end to end machine learning, playing with a variety of machine learning algorithms, and providing the best in class seamless experience, we want to hear from you.


Sparkflows offers a competitive salary, health benefits, great location, complementary meals and snacks etc. Join us!


Write to us at

Apache Spark & Big Data Engineer (Bay Area, California)

Job Description

We are looking for a highly skilled Apache Spark and Big Data engineer who is comfortable building big data applications. Big Data Engineers are responsible for designing and developing end to end big data applications. The engineers should be able to interact with data scientists, product managers and operations to make things successful.


Responsibilities and Duties

  • Design, develop and test new modules throughout their life cycle.

  • Collaborate with team in defining architecture and design of new modules.

  • Deep dive to find performance bottlenecks.

  • Work with Q/A team to develop and maintain regression and unit testing platform.

  • Build big data applications and pipelines on Azure, GCP and AWS.

  • Follow coding conventions, policies and procedures provided.

Qualification and Skills

  • Experience in building big data applications with Apache Spark.

  • Experience in writing and analyzing Hive queries.

  • Demonstrated experience of 3+ yrs in Big data technology stack - Hadoop, HDFS & Spark.

  • Expert in Scala or Java. Experience working on Linux based systems.

  • Experience in performance of Big Data processing.

  • Proficiency in Computer Science fundamentals - Object oriented design, Data structures & Algorithms.

  • Collaboration to work with cross functional team (Test, deployment & support) to make things happen.