img
Permanent

Senior Data Engineer

London
money-bag Negotiable
Posted 1 week ago

Position: Senior Data Engineer

Salary: Negotiable, depending on Experience

Location: Hybrid - Bristol, Manchester, or London
Security Requirement: SC Clearance to start and be willing / able to obtain DV Clearance

We are seeking Data Engineers, with a keen interest / experience in AI /ML and strong proficiency in Python Scripting. To join our Consultancy client working on meaningful projects in the Defence and Security sector.

Required Experience:

  • End-to-End Data Development: Strong experience with data pipelines, ETL processes, and workflow orchestration, demonstrating best practices across tech stacks.
  • Strong Python Skills
  • Diverse Data Handling: Familiarity with batch, streaming, real-time, and unstructured data sources.
  • Architectural Design & Systems Thinking: Ability to design and build scalable, high-performance data solutions.
  • Data Modelling & Warehouse Design: Proficiency in data modelling, warehouse design, and database optimization, with examples of logical and physical models.
  • Distributed Data Systems: Experience in deploying, managing, and tuning distributed systems for optimal reliability and performance.
  • Coding & Development Practices: Demonstrated coding expertise with modular, reusable, and efficient code in various languages.
  • Development Lifecycle: Understanding of SDLC, CI/CD pipelines, and version control.
  • Data Governance & Security: Knowledge of data security, governance, metadata management, and master data principles.

Key Responsibilities:

As a Senior Data Engineer, you''ll bridge the gap between client needs and technical solutions, creating data pipelines that ingest, transform, and enrich large data volumes. You''ll have client-facing responsibilities, delivering high-quality data solutions in multi-disciplinary teams across industries.

In this consulting role, your responsibilities will vary depending on client engagement focus and your skillset, but will often include:

  • Data Engineering & AI Integration: Apply data engineering tools, integration frameworks, and query engines to create high-quality, standardised data for AI applications and reporting.
  • Data Pipeline Development: Design and implement robust data pipelines and stores in collaboration with other engineers and developers.
  • Innovative Problem Solving: Bring fresh approaches to challenging data engineering problems.
  • Architecture for Scale: Design scalable, complex data architectures that provide cross-team value.
  • Data Modelling & Governance: Establish standards in logical and physical data modelling and data governance.
  • Distributed Computing: Employ parallel processing, streaming, and batch workflows to manage large data volumes effectively.
  • ETL & Workflow Automation: Build ETL processes and automated workflows for efficient data movement.
  • System Optimization: Tune data systems for performance, scalability, and monitoring.
  • Data Security: Apply best practices for information security, including encryption and data anonymity for sensitive data assets.
  • Data Governance & Quality: Manage metadata, data lineage, and data quality standards.

If you''re passionate about using data engineering and AI to solve complex problems in the Defence and Security sector, we''d love to hear from you!

People Source Consulting Ltd is acting as an Employment Agency in relation to this vacancy. People Source specialise in technology recruitment across niche markets including Information Technology, Digital TV, Digital Marketing, Project and Programme Management, SAP, Digital and Consumer Electronics, Air Traffic Management, Management Consultancy, Business Intelligence, Manufacturing, Telecoms, Public Sector, Healthcare, Finance and Oil & Gas.

Other jobs of interest...

CV-Library
City of LondonYesterday
money-bag£55000 - £75000/annum
CV-Library
City of LondonYesterday
money-bag£55000 - £75000/annum
CV-Library
City of LondonYesterday
money-bag£55000 - £75000/annum
CV-Library
LondonYesterday
money-bag£55000 - £75000/annum

Perform a fresh search...

  • Create your ideal job search criteria by
    completing our quick and simple form and
    receive daily job alerts tailored to you!

Jobs. Straight to your inbox!