Data Engineer

Data Engineer6 Month ContractLocation: London (Onsite 4 days a week)Rate: £700 - £800 per day (INSIDE IR35)Skills: Python, Azure, Kubernetes, Apache Airflow, Apache Spark, DV ClearanceWe are looking to recruit a DV Cleared Data Engineer for a leading technology consultancy. This will be joining an established defence programme requiring additional engineering support across an existing data platform.You will be responsible for building, maintaining and improving new and existing data pipelines within a secure MoD environment.Due to the nature of the work you will be required to work on site in London 4 days a week.Key ResponsibilitiesBuild and maintain robust data pipelines using PythonWork across Azure environments on the low sideSupport data engineering workloads within Kubernetes environments on the high sideDevelop and maintain workflows using Apache AirflowWork with Apache Spark for large-scale data processingSupport the ongoing development and improvement of established data pipelinesHelp ensure data remains within MoD-owned infrastructure in line with data governance requirementsWork within an existing long-running programme requiring additional delivery capacityEssential SkillsActive DV ClearanceStrong Python data engineering experienceExperience with AzureStrong knowledge of KubernetesExperience with Apache AirflowExperience with Apache SparkStrong background building and maintaining production data pipelinesDesirablePrevious experience working within a Palantir ..... full job details .....