In short
Senior Big Data Engineer in Singapore designing, building, and optimizing data pipelines on Hadoop/Cloudera and Spark, with Python Django microservices and Redis, Hive, and Kafka for streaming and warehouse work, collaborating closely with data scientists.
About the role
QUESS ENGINEERING PTE. LTD. is seeking an experienced Big Data/Data Engineering professional in Singapore to design, build, and optimize data workflows using Hadoop/Cloudera and Spark.
You will develop Python Django-based microservices and leverage Redis, Hive, and Kafka for reliable data processing. The role requires deep expertise in data warehouse management, streaming analytics, and collaboration with data scientists to drive business initiatives.
Requirements
- Experience as a Big Data/Data Engineering professional
- Design, build, and optimize data workflows using Hadoop/Cloudera
- Experience with Spark/PySpark
- Develop Python Django-based microservices
- Experience with Kafka
- Experience with Redis
- Experience with Hive
- Deep expertise in data warehouse management
- Deep expertise in streaming analytics
- Collaboration with data scientists
Nice to have
- Experience with Airflow (from job title)