Responsibilities

• Meet with business unit representatives to evaluate business automation needs
  
• Perform analysis based upon business requirements
  
• Participate in systems analysis, general systems design, and detail systems design
  
• Write or update technical documentation including data flows
  
• Develop new or modify existing programs based on business requirements while following departmentâ??s standards and guidelines
  
• Work with applications systems designs and specifications and utilize standards, procedures, and techniques
  
• Generate, test, and document program code
  
• Research and resolve problems in the production environment as needed
  
• Work as a team member in the development of complex programs
  
• Assist others with debugging as needed
  
• Learn new tools, technologies, and methodologies as appropriate
  
• Learn user functions, organization, and role in the enterprise
  

Qualifications

• 2+ years of Hadoop experience with any distribution Hortonworks HDP or Cloudera CDP or IBM BigInsight
  
• 2+ years of ETL experience and proven understanding of Hadoop, Spark and Hive
  
• 2+ experience in using file formats like XML, JSON, Avro, Parquet, Sequence, ORC and text files
  
• 2+ years of experience in workflow/schedulers like Oozie
  
• 5+ years of programming experience using Scala or Python or Java with the ability to pick up new languages and technologies quickly
  
• 5+ years of experience in Relational Databases such as MS SQL Server, IBM Netezza
  
• Demonstrated ability to create or modify SQL queries of medium to high complexity on MS SQL Server or IBM Netezza platform
  
• At least one implementation experience in Enterprise Data Lake solution and processing large amounts of structured, semi-structured and unstructured data at scale
  
• Good understanding and experience in multi-threading and concurrency concepts
  
• Ability to work efficiently under Unix/Linux environment
  

Nice To Have

• Experience in Big data implementation in cloud (IaaS or PaaS)
  
• Experience in NoSQL Cassandra or MongoDB or HBase or similar
  
• Experience in Hadoop SQL engine such as Impala or BigSQL
  
• Unix/Linux Shell Scripting
  
• Experience in real-time data processing using Flume or Kafka
  
• Experience in Informatica Big Data Management ETL tool
  
• Experience mentoring team members