Lead Data Engineer

5 days ago


UK, UK, United Kingdom Recann Full time

Lead Data Engineer


We are on the lookout for an experienced Lead Data Engineer to join our customer's innovative team on a contract basis.


Key Responsibilities:

  • Provide strategic leadership to the Data Engineering team, fostering best practices in software development to create robust AWS Glue data pipelines.
  • Oversee the secure management of the data engineering lifecycle, ensuring all pipelines are designed, built, deployed, and maintained with security in mind.
  • Take an active role in developing data pipelines by writing, testing, and deploying scalable and efficient code.
  • Construct CI pipelines and implement Infrastructure as Code for deployment across AWS environments.
  • Establish and configure development environments and services, including databases and various AWS components.
  • Advocate for Agile and DevOps methodologies to streamline software development processes and enhance delivery speed.
  • Facilitate daily stand-up meetings to assess team progress, address challenges, and ensure alignment with project goals.
  • Conduct thorough code reviews and maintain comprehensive technical documentation.
  • Collaborate with stakeholders to gain insights into existing data workflows and support development efforts.
  • Mentor and support team members through guidance, pair programming, and code review sessions to help them achieve their potential.


Location: Hybrid (Greater Manchester - 2 days per week)


Ideal Candidate:


To thrive in this role, you will need:


  • Extensive experience in leading the development of AWS Glue data pipelines, with a strong background in similar roles within medium-sized organizations.
  • A minimum of four years’ experience in data engineering, showcasing your ability to write clean, testable, and maintainable code.
  • Proficiency in AWS technologies, including Glue, Lambda, S3, as well as programming languages such as Python and SQL, and familiarity with PostgreSQL and PySpark.
  • Knowledge of building CI pipelines, preferably using Microsoft Azure DevOps, alongside a solid understanding of the software development lifecycle and Agile methodologies (Scrum and Kanban).
  • Experience with DevOps principles and practices, as well as lean development methodologies.
  • A strong grasp of information security requirements, particularly regarding GDPR compliance and secure development lifecycles.


Skills and Competencies:


  • Exceptional verbal and written communication skills to effectively engage with technical and non-technical stakeholders at all levels.
  • Strong people management abilities, creating a positive team environment, resolving conflicts, and being responsive to team needs.
  • Demonstrated decision-making capabilities and the ability to quickly identify requirements to deliver effective solutions without unnecessary complexity.
  • A detail-oriented mindset with excellent problem-solving skills, able to prioritize tasks under pressure and maintain a focus on quality.
  • A commitment to continuous learning and adapting to new technologies, methodologies, and best practices.



Apply Today for further information.


  • Lead Data Engineer

    2 months ago


    UK, UK, United Kingdom Noir Full time

    Lead Data Engineer (Python, PySpark) - Remote(Lead Data Engineer, Team Lead, Technical Lead, Senior Data Engineer, Data Engineer, Python, PySpark, SQL, Big Data, Databricks, R, Machine Learning, AI, Agile, Scrum, TDD, BDD, CI / CD, SOLID principles, Github, Azure DevOps, Jenkins, Terraform, AWS CDK, AWS CloudFormation, Azure, Lead Data Engineer, Team Lead,...

  • Lead Data Engineer

    2 months ago


    UK, UK, United Kingdom Head Resourcing Full time

    Lead Data Engineer - £75k plus bonus & benefits - hybrid 2 days in ManchesterAn exciting opportunity has arisen to join this leading household name, based in Manchester city centre (hybrid).The role will be critical in delivering and ensuring data quality, consistency, and reliability working closely with other data engineers in the team as well as the...

  • Data Lead

    6 days ago


    UK, UK, United Kingdom InfoSec People Ltd Full time

    I am delighted to be continuing our work with a fantastic Cyber Security start-up company. We are looking for a Data Engineering Lead who will bring leadership and expertise to the way we exploit data collected by our award-winning security awareness product. We’re looking for an experienced data practitioner who’s as comfortable implementing data...

  • Lead Data Engineer

    1 week ago


    London, UK, UK, United Kingdom Harnham Full time

    Lead Data Engineer London - Hybrid (3 days per week in office) Up to £90,000 + BenefitsOVERVIEWWe are partnering with a prominent investment company within the infrastructure sector, seeking a skilled Senior Data Engineer to help establish a data strategy and support long-term data-driven growth. With recent consulting insights highlighting the need for a...

  • Lead Data Scientist

    2 weeks ago


    UK, UK, United Kingdom Rise Technical Full time

    Lead Data Scientist (Crypto)UK - Remote£100,000 to £150,000 + Company Equity + Fully RemoteExcellent opportunity for a Lead Data Scientist with experience in crypto to join a fast-growing Fintech that has recently gained Series A funding, to lead the backend development and systems across the organisation.This company are a B2B Fintech company specialising...

  • Data Engineering Lead

    2 weeks ago


    London, UK, UK, United Kingdom Datatech Analytics Full time

    Data Engineering Lead – AWS & SnowflakeHybrid working: 3 days in TW6, Middlesex offices & 2 days homer/remoteSalary: Negotiable to £70,000 DOE plus 40 % bonus potentialJob Reference: J12869Full UK working rights required/no sponsorship availableTHE ROLELooking for a challenge in one of the world’s largest airfreight logistics organisation and a FTSE 100...

  • Data Engineer

    2 months ago


    UK, UK, United Kingdom Eames Consulting Full time

    We are delighted to be working with a global speciality Insurance firm that are looking to expand there Pricing Data engineering team with an experienced Data engineer.Responsibilities:Build and maintain data pipelines in a Spark-based platform to support the Pricing function.Develop solutions that enhance Global Pricing’s data-driven approach, leveraging...

  • Senior Data Engineer

    2 weeks ago


    UK, UK, United Kingdom Understanding Recruitment Full time

    Senior Data Engineer | 1x per month in the London office (flex)I am looking for a Senior Data Engineer to join a cutting edge start-up that’s leading the charge in IoT, and data-driven tech across Europe!They are revolutionising the automotive space with cutting-edge in-car technology, web apps, and APIs. To accelerate the building of their greenfield...

  • Lead Data Scientist

    1 month ago


    UK, UK, United Kingdom Rise Technical Full time

    Lead Data Scientist (Crypto)UK - Remote£100,000 to £150,000 + Company Equity + Fully RemoteExcellent opportunity for a Lead Data Scientist with experience in crypto to join a fast-growing Fintech that has recently gained Series A funding, to lead the backend development and systems across the organisation.This company are a B2B Fintech company specialising...


  • UK, UK, United Kingdom Vantage Data Centers Full time

    About Vantage Data CentersVantage is committed to being a workplace of inclusion, equity, respect and acceptance. We celebrate diversity and intentionally seek out opportunities to learn from one another’s experience.Vantage Data Centers powers, cools, protects and connects the technology of the world’s well-known hyperscalers, cloud providers and large...

  • Data Engineer

    1 month ago


    UK, UK, United Kingdom Cognify Search Full time

    Senior Data EngineerJoin a London Based B2B SaaS start up utilising data to drive insights! Salary: Up to £100,000 Benefits: Stock Options Industry/Type of business: SaaS Workplace: Hybrid (3 days a week in London)Would you be interested in being part of a dynamic company transforming how businesses develop strategy and execute it?The team has created a...

  • Data Engineer

    1 week ago


    UK, UK, United Kingdom Next Phase Recruitment Full time

    Description‍The client is seeking software engineers with a strong background with Python programming and ideally a chemistry or material science background to develop domain-specific applications. Their goal is to revolutionize industrial research and development. They’re looking for motivated engineers who can help to build out tools that will...

  • Graduate Data Engineer

    2 months ago


    UK, UK, United Kingdom Oho Group Ltd Full time

    Graduate Data Engineer - Fast Growing FinTech - ex Facebook / Google A very fast scaling FinTech in London founded by ex Facebook and Google Engineer are looking for a talented Data Engineer to join their team. In this role as a Data Engineer you’ll work with extremely high calibre engineers and hit the ground running from day one, starting off with where...

  • Senior Data Engineer

    1 month ago


    UK, UK, United Kingdom SPG Resourcing Full time

    Job Title: Microsoft Fabric SME / Senior Data Engineer (3-Month initial Contract)Location: Remote Day Rate: £500-£600 (Outside IR35) About the Role:We are seeking a Microsoft Fabric SME/ Senior Data Engineer to join one of our clients for a critical 3-month project within the manufacturing sector. You will play a key role in the design, setup, and scaling...

  • Senior Data Engineer

    3 months ago


    UK, UK, United Kingdom TLP Consultancy Ltd Full time

    £65,000 to £75,000 Permanent Near Epsom, Surrey (location is most easily reached by own transport) Hybrid working (3 days a week in the office) Generous benefits package (Performance related bonus, Pension, access to their car scheme, Birthday voucher, Workplace Nursery Benefit & Cycle Scheme, holiday purchase scheme, discounted onsite gym membership,...

  • Data Engineer

    2 days ago


    UK, UK, United Kingdom FBSPL Full time

    We're going to start by talking about our exciting new Data Engineer position, but if you'd like to know more about the company then just reach out and we’ll fill you in!You will be working in our Data Engineering department to support every part of the business in their journeys to streamline and automate the data they have. You will lead the way...

  • Lead Java Engineer

    2 weeks ago


    UK, UK, United Kingdom Understanding Recruitment Full time

    Lead Java Engineer – Vehicle Telematics/Diagnostics – Remote – **£95K + Bonus**We are excited to be partnering once again with a leading innovator in the automotive sector, marking our second year of successful collaboration. Our client is at the forefront of developing an advanced platform designed to predict vehicle reliability issues, leveraging...

  • Senior Data Engineer

    1 month ago


    UK, UK, United Kingdom Tech4 Full time

    Senior Data Engineer - Python, ETL, AWS - health tech - tech for good, make a positive impact on the world. Highly successful and fast growing organisation.This is a new role and part of the Technology team. The Senior Data Engineer will design, build, maintain, and troubleshoot the systems and infrastructure that enables the organisation to collect, store,...


  • UK, UK, United Kingdom The MWek Company Full time

    My client, a growing Fintech, is looking for a Financial Data Engineer to apply critical thinking and data engineering expertise to enhance our financial data systems and pipelines.Qualifications* Master’s degree in Computer Science, Data Engineering, Quantitative Finance, or a related technical field* A minimum of 1 year of experience is required, with...


  • UK, UK, United Kingdom trg.recruitment Full time

    Lead Analytics Engineer-GCP, Bigquery, dbtWant to help save lives?*Hybrid in Central London**2 stage interview*Did you know a frontline worker loses their life every 10 seconds?Imagine leading a team in developing a risk management platform to help save lives.Your role is crucial in creating and maintaining an innovative mobile application that will be used...