In short
Senior data engineer building end-to-end big data processing systems — ingestion, data lakes, ETL, data warehouses — using Spark, Spark SQL and cloud-native services, with multi-cloud deployments. Also leads and mentors the team and needs client-facing communication skills.
About the role
1. Experience in object oriented design and implementation
2. Experience in creating end-to-end data processing systems - Design, build, operate, test, deploy.
3. Experience in building big data solutions using Spark, Spark SQL and associated technologies - databases, streaming platforms, data warehouses etc.
4. Experience in creating lean and efficient processes to track work and project progress
5. Highly focused on deliverables, identifying blockers that effect the delivery and resolve that with the team members
6. Good coding and debugging skills
7. Experienced team leader and mentor
8. Good knowledge on creating efficient prodcution deployments for data projects leveraging the native services of specific cloud providers.
9. Experience in multi cloud deployments is an advantage
10. Experience building enterprise data platforms: Data Ingestion, Data Lake, ETL, Data Warehouse
11. Good communication and articulation skills for client facing.
12. Nice to have Big Data technologies: Hadoop, HDFS, Hive, Impala, HBase, MongoDB, Cassandra, Kafka
13. Domain specific knowledge is a big advantage - Health care, BFSI, Retail etc.NA
Requirements
- Experience in object-oriented design and implementation
- Experience creating end-to-end data processing systems (design, build, operate, test, deploy)
- Experience building big data solutions using Spark, Spark SQL and associated technologies (databases, streaming platforms, data warehouses)
- Experience creating lean, efficient processes to track work and project progress
- Highly focused on deliverables, identifying and resolving blockers with team members