About the role
Together Staffing group is seeking an experienced Python/Spark/AI Developer to replatform legacy T-SQL workloads onto Spark and Delta Lake, while building robust APIs and AI-enabled capabilities. The role suits an autonomous developer delivering production-quality code in a specification-driven environment.
The candidate will design and implement Spark pipelines, translate T-SQL logic to Spark SQL, and contribute to testing, documentation, and ADR-driven architecture.
Build production-grade Spark/PySpark pipelines with Delta Lake. Replatform legacy T-SQL logic into Spark SQL with parity. Develop robust, type-safe Python with strong testing. Create REST APIs using FastAPI or similar frameworks. Implement authentication, idempotency and reliable job-status semantics. Validate migrated pipelines against legacy systems with evidence of parity. Work from ADRs and design documents. Contribute tests and documentation alongside code changes. Engage in PR-driven development and CI processes. 4+ years of professional Python development. 2+ years of production Spark/PySpark experience. Strong Python 3.12 with type hints and Pydantic. Strong Spark/PySpark knowledge and Spark SQL expertise. Proficient in translating legacy T-SQL to Spark SQL and handling MERGE operations. Experience with Delta Lake or similar lakehouse tech (Iceberg/Hudi). Testing experience with pytest and CI/CD practices. REST API development experience, preferably with FastAPI. Experience with Docker-based development environments and Git workflows. Bachelor's degree in Computer Science, Engineering or equivalent. Ability to work independently from written specs and requirements.