About the role
You will bring card transaction, settlement, ledger and onchain data into the warehouse and keep it correct. You will build idempotent batch and streaming pipelines that handle late and duplicate data, own ingestion, build dbt models, orchestrate in Dagster and add quality checks and alerts.
Responsibilities
- Build scalable batch and streaming pipelines
- Process events near real time with deduplication, windowing and late or out-of-order handling
- Own batch ingestion from databases (CDC), files (SFTP and GCS), APIs and onchain sources
- Build incremental dbt models and reconcile streaming and batch outputs
- Orchestrate pipelines in Dagster with sensors, schedules and alerts
- Add freshness, volume and reconciliation checks to every source
- Work with backend pods on event and table contracts
- Partner with Analytics Engineers, Data Scientists and Risk on data needs
Requirements
- At least 3 years building production data solutions, owning batch and streaming pipelines
- Experience designing data solutions at scale
- Strong SQL and Python / Rust, plus cloud data warehouse experience such as BigQuery, ClickHouse or Snowflake
- Streaming experience with Pub/Sub, Kafka, Dataflow/Beam or Flink
- Batch experience with dbt and Dagster or Airflow
- Production data quality testing, monitoring and reconciliation
- Ability to debug across source, pipeline, warehouse and report
- Ability to use AI tools while checking output
Benefits
- Flexible time off (minimum 10 days)
- Flexible working with home workspace stipend
- 95% health, dental and vision cost coverage for employees and 90% for dependents (US)
- 100% company-subsidized life insurance
- 401(k) with 4% company match
- Equity option plan
- Bonus
- Monthly health and wellness stipend
- In-office meals via DoorDash credit
- Team summits and offsites
Requirements
- At least 3 years building production data solutions, owning batch and streaming pipelines
- Experience designing data solutions at scale
- Strong SQL and Python / Rust, plus cloud data warehouse experience such as BigQuery, ClickHouse or Snowflake
- Streaming experience with Pub/Sub, Kafka, Dataflow/Beam or Flink