Platform Data Engineer
New York / New Jersey , NY (On-Site)
Job Description:
JD-
Cloud Devops, python, Big Data Admin exp on AWS , AWS lake formation ( lake house vs Data fabric ), Data Product concepts, spark jobs auditing, monitoring and Instrumentation, spark perf tuning, exp in creating service and tools to manage multi tenant env,
Tech stack ( Must )– terraform, knowledge of spark internals ( spark 3+ features ) – architecture, events, system metrices , AQE, job level resource management, Hudi, Parquet , IAM, AWS Glue, HIVE .
Good to have - Iceberg, airflow, terragrunt, dynamo DB.
Key Skills:
- Devops AND AWS
AWS, DevOps, Hadoop