Data Engineer Level I

DWP. Digital with Purpose We need a Data Engineer to join our Debt Solutions team and help shape the future of data at DWP. You''ll need experience of SQL-based data transformation, ETL pipeline development, and building user-facing dashboards and reports Our DWP Digital teams use fresh ideas and leading-edge tech to build and maintain digital solutions that will be used by nearly every person in the UK, every day and at key moments in their lives. DWP is the UK''s largest government department. We help people into work and make payments worth over £195bn a year to support and empower millions of people. The scale of what we do is extraordinary, and our purpose is unique. We''d love you to join us. What skills, knowledge and experience will you need? Experience developing and managing PySpark workflows within AWS environments. Knowledge of AWS Lambda and event-driven solutions to support automated data processing. Experience provisioning and supporting cloud infrastructure using Amazon EC2 and automation tools. Understanding of Agile ways of working and experience contributing to CI/CD delivery practices. Ability to optimise data processing, storage, and queries to improve performance and data quality. Experience designing and automating ETL pipelines using PySpark and Spark SQL.A strong full stack development background with technologies that include .NET 4,8, C#, JavaScript, C++ and Java. You and your role As a Data Engineer in DWP Digital, you''ll design, build ..... full job details .....