Data Engineer

As our next Data Engineer, you''ll be managing and developing data pipelines that transform raw data into valuable insights for Roke''s National Security customers, enabling downstream analytics and reporting. You''ll be working with diverse data sources (batch, streaming, Real Time and unstructured), applying distributed compute techniques to handle large datasets.What You''ll Do* Data pipeline development - Data ingestion and pipeline orchestration design and tooling.* Database schema design/database modelling* Data integration - Integrating and enriching data from various sources, ensuring data consistency and quality.* ETL processing design and coding - Extract transform and load processing such as NiFI.* Create code that is open by default and easy for others to reuse* Maintain and develop existing architectural components including Data Ingest and Data Stores* Work as part of an operational team investigating and diagnosing problem identified with integrated (enriched) data.* Explain the difference between user needs and the desires of the user* Data security - Implementing data security measures to protect sensitive information.* Be able to help the scrum team decompose user requests and key results into epics and stories.* Writing clean, secure code following a test-driven approach* Create code that is open by default and easy for others to reuse* Monitor and maintain - Monitor data systems for performance issues and make any necessary updates.Reasonable ..... full job details .....