In short
Data Quality Engineer at Shopee in Singapore building and maintaining layered Hive/Spark SQL data models across ODS, DWD, DWS, and ADS, plus production batch ETL pipelines with incremental loads, backfills, and partner onboarding. Core stack: Spark, Hive, HDFS, SQL, Python/Java.
About the role
SHOPEE IP SINGAPORE PRIVATE LIMITED is seeking a data engineer to design, build, and maintain layered Hive/Spark SQL data models across ODS, DWD, DWS, and ADS, including shared dimensions. You will develop production batch ETL pipelines on Spark and Hive, onboard partners, and manage incremental loads and backfills.
The role requires a Master’s degree in CS/Software/AI and hands-on experience with Spark, Hive, HDFS, and data-warehouse concepts, plus strong SQL and Python/Java skills.
Requirements
- Master's degree in CS/Software/AI
- Hands-on experience with Spark
- Hands-on experience with Hive
- Hands-on experience with HDFS
- Data-warehouse concepts
- Strong SQL skills
- Python and/or Java skills
- Design, build, and maintain layered Hive/Spark SQL data models (ODS, DWD, DWS, ADS)
- Develop production batch ETL pipelines on Spark and Hive
- Manage incremental loads and backfills