Big Data Engineer

Job Description

Date: Dec 5, 2018

Big Data Engineer 


This role requires an experienced Big Data developer with hands-on experience in implementing predictive analytics/machine learning/data science solutions on cloud based/distributed enterprise scale data lake platforms


Education: B.Tech/MS  in Computer Science, Engineering, Data Science, Operations Research, Business analytics, related fields.

  • Programming skills: Java, SCALA, Python
  • Data Ingestion: Sqoop, Flume, Nifi, Stream sets, Logstash
  • Data processing: Hive, Spark
  • Messaging system: kafka, Strom
  • Scheduling and workflows: Oozie, Azkaban, and Luigi
  • No SQL databases: MAPR, Hbase, Casandra, Mongo DB
  • Exposure to cloud deployment models

Other responsibilities:

  • Aid product development teams in coming up with new ideas for machine learning applications.
  • Understand necessary data requirements to implement machine learning applications.
  • Deploying machine learning solutions into production.
  • Optimizing solutions for performance and scalability.
  • Data engineering, i.e. ensuring a good data flow between database and backend systems.