About the role
You will design, build and run the systems behind data products, from ingestion to serving, owning your work from design document to production. You will build pipelines for blockchain data across 30+ chains, write backend services in Go and Python, model and tune ClickHouse workloads, build Databricks pipelines, solve performance problems, and mentor engineers.
Responsibilities
- Design and build scalable high-throughput data pipelines for blockchain data across 30+ chains
- Build backend data services and loaders in Go and Python
- Design and evolve data models and analytical workloads in ClickHouse
- Build incremental idempotent pipelines on Databricks
- Improve scale, reliability, freshness, performance and cost efficiency
- Investigate performance and reliability problems
- Work with product, analytics and investigations teams
- Help set technical direction through architecture, design and code reviews
- Mentor engineers and set operational practices
Requirements
- 5+ years building and operating production data systems including ETL and ELT
- Expert Python and strong SQL
- Go for backend services
- ClickHouse in production
- Databricks in production (PySpark, Delta Lake, Unity Catalog)
- Experience with billions of rows and tens of terabytes
- Kafka, Airflow and Kubernetes in production, Docker, Helm
- Google Cloud preferred, AWS fine
- Operations and communication mindset, runbooks, design documents
Benefits
- Health insurance
- Flexible time off
- Learning and development initiatives
- Work/life balance hours
- Team-building sessions
- Equity
Requirements
- 5+ years building and operating production data systems including ETL and ELT
- Expert Python and strong SQL
- Go for backend services
- ClickHouse in production
- Databricks in production (PySpark, Delta Lake, Unity Catalog)
- Experience with billions of rows and tens of terabytes