Data Engineer

ESSENTIAL ROLES and RESPONSIBILITIESIdentify and understand customer data-centric use cases within regulated financial services environmentsDesign and implement data ingestion, processing, and transformation pipelines on AzureBuild and maintain data pipelines for cleaning, normalisation, enrichment, and preparationApply appropriate data modelling techniques and architecture patterns, with a strong focus on medallion architectureOrchestrate, monitor, and optimise Azure Databricks jobs and Azure Data Factory pipelines across development, UAT, and production environmentsConfigure platforms, clusters, and compute resources to optimise performance, cost, and reliabilityUse automated CI/CD pipelines to manage, deploy, and version data artefacts and pipelinesOperationalise workflows developed by analysts and data scientistsSupport customers in adopting Azure data, analytics, and machine learning servicesEnsure secure storage, processing, and quality of customer dataEnsure networking and security best practices are applied when designing and operating data solutionsDesign solutions for processing large volumes of data using batch and streaming approachesCollaborate with analytics teams on data visualisation best practices and reporting enablementEnsure all solutions are well-documented, including pipelines, schemas, transformations, and operational runbooksGOVERNANCE and REPORTINGMaintain accurate documentation of data pipelines, schemas, transformations, and deployment ..... full job details .....