We're seeking a Senior Data Engineer to lead the technical delivery of a complex migration to Databricks on Azure. This role involves designing scalable pipelines, owning CI/CD for data, and mentoring the engineering team. The ideal candidate will have experience with Databricks, Azure, and CI/CD workflows.
Key Highlights
Technical Skills Required
Benefits & Perks
Job Description
Data Engineer / Senior Level / Up to £70,000 / Remote (UK) / 6-month FTC
We’re recruiting a Senior Data Engineer to lead the technical delivery of a complex migration to Databricks on Azure. This is a senior, hands-on leadership role: you’ll design scalable pipelines, own CI/CD for data, and mentor the engineering team.
Why this role?
You’ll shape architecture and best practice for a multi-phase migration - re-platforming legacy ETL/ELT, optimising performance, and introducing robust CI/CD and observability across data products.
Day-to-day
- Architect and deliver large-scale ETL/ELT pipelines on Databricks + Azure.
- Lead migration planning and execution: re-platforming legacy jobs, validating outputs.
- Implement CI/CD workflows for notebooks, jobs and infra (testing, release automation).
- Drive observability, monitoring and data quality practices.
- Mentor junior/mid engineers, define standards, and liaise with stakeholders.
Must-haves
- Experience as a data engineer with strong cloud data platform delivery.
- Deep Databricks experience and Azure data ecosystem expertise.
- Proven experience implementing CI/CD for data workloads.
- Strong Python, SQL, and production debugging skills.
- Experience leading technical workstreams and coaching others.
Salary: Up to £70,000
Location: Fully remote (UK-based)
Similar Jobs
Explore other opportunities that match your interests
Animal Friends
Senior Data Scientist
booksy
