Lead Databricks Engineer - Hybrid - Permanent

Lead Databricks Engineer - Hybrid - Permanent Overview We are seeking an experienced Lead Azure Databricks Engineer to design, build and optimise our enterprise data platforms. The role requires deep technical expertise, strong delivery capability and experience working in highly regulated environments within the Lloyd''s of London market.You will play a key role in shaping our cloud data platform, enabling advanced analytics and ensuring secure, scalable and reliable solutions across the organisation. This is a fully hands-on engineering role.Key responsibilitiesDesign and build enterprise data platforms using Microsoft Azure services including Azure Data Factory, Azure Data Lake Storage, Azure Key Vault, Azure Functions and Azure DatabricksDevelop scalable pipelines using Delta Lake, PySpark and Unity CatalogWork with underwriting, actuarial, delegated authority, bordereaux, exposure management, reinsurance, finance and risk teams to deliver solutions aligned to Lloyd''s market requirements and regulations including Solvency IITranslate business requirements into scalable cloud data solutions with architects, SMEs and product ownersLead the resolution of complex technical challenges and drive deliveryPromote best practices in data engineering, cloud architecture, CI/CD and data life cycle managementProvide technical leadership and mentoring to engineersImprove platform performance, resilience, cost efficiency and observabilityDeliver reliable, high quality data pipelines ..... full job details .....