Data Engineer

Location

Glasgow or Newcastle-upon-Tyne

About the Company

Are you a highly skilled Data Engineer (SEO) ready to combine your technical leadership with a meaningful mission? The Infected Blood Compensation Authority (IBCA) is a new arm's-length body set up, at unprecedented pace, to administer compensation to people whose lives have been impacted by the infected blood scandal. IBCA will ensure payment is made in recognition of the wrongs experienced by those who have been infected by HIV, Hepatitis B or C, as well as those who love and care for them. They have been frustrated and distressed by the delays in achieving proper recognition, and we must help put this right. We are committed to putting the infected and affected blood community at the centre of every decision we make and every step we take to build our organisation to deliver compensation payments.

About the Role

This role will lead our data engineering capability within the engineering team of the Data Operations arm of the IBCA Data Directorate. The Data Operations team is responsible for developing and running safe and secure data solutions that provide a single source of truth for those going through their compensation journey. We are building a new data platform using Amazon Web Services (AWS) and data management and intelligence products using Databricks, Quantexa and Tableau. We are taking a product-centric approach treating data as a product and are building squads around our products, with a focus on paying compensation to those impacted by the infected blood scandal seamlessly.

Responsibilities

• Provide technical guidance for the development of robust, automated data pipelines and master data management processes for the data platform and products, encompassing DevSecOps best practices.
• Ensure tools and techniques are scalable, secure, and efficient.
• Be responsible for ensuring the right data engineering practices are embedded consistently and to industry standard best practices within the data platform delivery teams.
• Further develop your own data engineering and leadership skills.
• Work with business stakeholders and across digital service teams, understanding their needs and translating them into data development.

Qualifications

Strong experience of cloud-native data engineering in AWS. Experience building and maintaining complex data pipelines (both ETL and ELT) in a rapid delivery setting. Expertise covering data quality & transformation processes, data matching, and master data management.

Required Skills

• Experience working with structured and unstructured data and data lakes.
• Ability to service both operational and analytical business needs.
• Proficiency in writing clear, parameterised code in two or more of the following: Python, Databricks, Apache Spark (Pyspark, Spark SQL), NoSQL, Scala.
• Experience of delivering through Agile/DevOps working practices in multi-disciplinary teams.
• Strong problem-solving skills, including assessing and mitigating risks while identifying opportunities for innovation.
• Experience in the full end-to-end data lifecycle for design, build, and test.
• Demonstrable experience of setting up data engineering processes from scratch or implementing large changes to existing processes within an organisation.

Preferred Skills

• Familiarity with practices such as CI/CD, Scrum, and Automation.
• Knowledge of the interactions and dependencies with data architecture, data modelling, and testing engineering.
• Must operate within a cloud DevSecOps environment.

Additional information:

Working at IBCA gives you a huge opportunity to make an impact on those who deserve compensation, and this role suits a candidate who can lead a team of data engineers to deliver solutions from the ground up to take from ideation to reality so that data is an enabler to everything IBCA does.

A minimum 60% of your working time should be spent at your principal workplace. Although requirements to attend other locations for official business will also count towards this level of attendance.

Apply before 11:55 pm on Monday 19th January 2026