About the role
Job Title: Senior Big Data Engineer
Experience: 1015 Years
Location: Bangalore / Chennai
Job Summary
We are looking for an experienced Senior Big Data Engineer with strong expertise in the Hadoop ecosystem, PySpark, SQL, Linux, and Shell Scripting. The ideal candidate should have hands-on experience in designing, developing, and optimizing large-scale data processing solutions and data warehouse architectures.
Key Responsibilities
- Design, develop, and maintain scalable Big Data solutions using the Hadoop ecosystem and PySpark.
- Build and optimize large-scale data processing pipelines.
- Work on data warehousing concepts, architecture, and best practices.
- Develop, optimize, and troubleshoot complex SQL queries.
- Implement workflow orchestration, scheduling, and monitoring using Apache Airflow.
- Perform Linux administration tasks and Shell Scripting for automation.
- Ensure data quality, performance, reliability, and scalability.
Mandatory Skills
- Core Big Data Hadoop Ecosystem
- PySpark
- Advanced SQL
- Linux & Shell Scripting
- Data Warehousing Concepts and Architecture
- Apache Airflow – Workflow Orchestration and Monitoring
Good-to-Have Skills
- Kubernetes
- DevOps
- MinIO
- Apache Iceberg
Requirements
Nice to have
- Kubernetes
- DevOps
- MinIO
- Apache Iceberg