Lead Data Engineer

1 month ago


City of London, United Kingdom Cornwallis Elt Full time

Lead Data Engineer – Python, SQL, dbt, AWS, Dremio, Data Mesh, ETL, Data Pipelines, Data Warehousing, DWH, S3, Glue, Data Lake, Automation, London

A Lead Data Engineer is sought after by a leading workplace pensions provider, to run their expanding data engineering team in their London City office. The team have recently built out a next-gen data lake hosted in AWS which makes use of Dremio, Data Mesh architecture and a wide range of AWS tools (S3, Glue, Step Functions etc.).

With the infrastructure of this new Data Lake in place, there is now a focus on enhancing the data stored within (via monitoring, cleaning and, as well as a requirement to design, build and implement robust ETL pipelines for effective integration with the wider business.

As a Lead Engineer, you will also be responsible for overseeing a small team of Data Engineers, setting and driving best practises and standards, ensuring project delivery is kept on track and regularly engaging with stakeholders up to C-suite for feedback/updates.

This will also involve working directly alongside the Head of Data Platforms to ensure ongoing engineering work is aligned to overall architecture and roadmap, as well as future project planning.

If you demonstrate the following skillset, please do apply

  • 7/8 years’ experience in Python programming, particularly around data analysis, automation and the building of data pipelines (as well as an understanding of data warehousing concepts)
  • Strong SQL skills, especially in relation to ETL processes and data management (querying, cleaning, storing)
  • Experience working with and deploying to AWS cloud (having worked with any/all of S3, Glue, Lambda, Step Functions etc.)
  • Usage of PowerBI for data visualization
  • Experience of both Dremio and Data Mesh Architecture
  • Prior experience of leading software engineering teams (including line management)
  • Clear, confident communications with previous experience of working directly with business users


  • London, United Kingdom Data Idols Full time €90,000 - €110,000

    Lead Data Engineer Location: London - Hybrid We are currently looking for a Lead Data Engineer to join our dynamic and forward-thinking team at a prominent British Insurance Company. This role is integral to our operations as it involves both maintenance of existing systems and the development of new projects. As a Lead Data Engineer, you will report...


  • London, United Kingdom Data Idols Full time €90,000 - €110,000

    Lead Data Engineer Location: London - Hybrid We are currently looking for a Lead Data Engineer to join our dynamic and forward-thinking team at a prominent British Insurance Company. This role is integral to our operations as it involves both maintenance of existing systems and the development of new projects. As a Lead Data Engineer, you will report...


  • London, United Kingdom Data Idols Full time

    Lead Data Engineer Location: London - Hybrid We are currently looking for a Lead Data Engineer to join our dynamic and forward-thinking team at a prominent British Insurance Company. This role is integral to our operations as it involves both maintenance of existing systems and the development of new projects. As a Lead Data Engineer, you will report...


  • London, Greater London, United Kingdom Data Idols Full time

    Data Idols is looking for a Chief Data Engineering Lead to support the delivery of data transformation.As a key member of the team, you will collaborate with the wider data teams and drive high standards across the data function. The role involves building pipelines to deliver insights enabling the company to be more data driven in their decision making.Key...


  • London, Greater London, United Kingdom Data Idols Full time

    The OpportunityWe are looking for a skilled Data Engineering Manager who has extensive experience in managing teams, implementing data strategy, and working with Azure. As a hands-off role, you will have the autonomy to lead the team and drive results.Your key responsibilities include:Implementing data strategy to drive business valueManaging and mentoring...


  • City of London, United Kingdom Cornwallis Elt Full time

    Lead Data Engineer – Python, SQL, dbt, AWS, Dremio, Data Mesh, ETL, Data Pipelines, Data Warehousing, DWH, S3, Glue, Data Lake, Automation, London A Lead Data Engineer is sought after by a leading workplace pensions provider, to run their expanding data engineering team in their London City office. The team have recently built out a next-gen data lake...


  • City of London, United Kingdom Premier Group Recruitment Full time

    My Client a leading powerhouse within London are looking to add an New Lead Data Engineer to their team. The Data Engineer will support our software developers, PowerBI developers, data analysts and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects. Job Title: Lead Data...


  • City Of London, United Kingdom NTT DATA, Inc. Full time

    Competitive salary in London:">We are offering a competitive salary package for this Lead Site Reliability Engineer position.About the role:This is an exciting opportunity to join NTT DATA as a Senior SRE and lead our team in implementing site reliability engineering principles.The ideal candidate will have experience in designing and developing dashboards...


  • London, Greater London, United Kingdom Data Science Festival Full time

    Data Engineering Manager Role OverviewAt the forefront of our company's innovative landscape, we seek a seasoned Data Engineering Manager to spearhead data transformation initiatives. With an exciting tenure at one of the UK's most renowned retail brands, this pivotal role requires expertise in implementing data strategy and delivering value across the...


  • City of London, United Kingdom Cornwallis Elt Full time

    Lead Data Engineer Opportunity at Cornwallis EltWe are seeking an experienced and skilled Lead Data Engineer to join our team in London. The successful candidate will be responsible for overseeing the development of data engineering solutions, ensuring data quality, and leading a small team of data engineers.The ideal candidate will have 7-8 years of...


  • London, Greater London, United Kingdom Data Science Festival Full time

    About the RoleAs a Staff Data Engineer, you'll be responsible for building and maintaining complex data pipelines, collaborating with our wider data teams to deliver insights that drive business growth.The ideal candidate will have extensive experience with data pipelines, data warehousing, and cloud-based technologies such as Azure.


  • London, Greater London, United Kingdom Data Idols Full time £80,000 - £90,000

    About the RoleWe are currently seeking a Senior Data Platform Engineer to join our team at Data Idols, a renowned retailer in the digital-first era. This is an exciting opportunity to contribute to the design, implementation, and maintenance of a robust data platform essential to our ongoing digital transformation.Your Key Responsibilities:Design and...


  • London, Greater London, United Kingdom Data Science Festival Full time

    **Job Overview**Data Science Festival is seeking an experienced Big Data Engineering Lead to join their team in London – hybrid. This is a fantastic opportunity for a skilled professional to drive data transformation and deliver value throughout the company.This hands-off role requires someone with extensive experience in managing teams, stakeholder...


  • London, Greater London, United Kingdom Data Science Festival Full time

    Data Engineering Manager OpportunityWe are seeking an experienced Data Engineering Manager to lead our team in implementing data strategy and delivering value throughout the company.The RoleThe successful candidate will have extensive experience in managing teams, stakeholder management, and implementing data strategy. They will work closely with the teams...


  • City Of London, United Kingdom Crimson Limited Full time

    We are looking for a seasoned data engineer to join our team as a Lead Data Solutions Engineer.Responsibilities:Develop and maintain complex data applications using Python, Spark, and other technologies.Design and implement data pipelines and APIs to support data integration and analytics.Leverage expertise in data infrastructure and systems to optimize data...


  • City of London, United Kingdom Cornwallis Elt Full time

    Cornwallis Elt is seeking a Lead Data EngineerWe are looking for an experienced and skilled data engineer to lead our expanding team in London. The ideal candidate will have strong leadership skills, as well as experience with Python, SQL, AWS, Dremio, and Data Mesh Architecture.The successful candidate will oversee the design and implementation of robust...


  • City of London, Greater London, United Kingdom Premier Group Recruitment Full time

    My Client a leading powerhouse within London are looking to add an New Lead Data Engineer to their team. The Data Engineer will support our software developers, PowerBI developers, data analysts and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects. Job Title: Lead Data...


  • City of London, Greater London, United Kingdom Premier Group Recruitment Full time

    My Client a leading powerhouse within London are looking to add an New Lead Data Engineer to their team. The Data Engineer will support our software developers, PowerBI developers, data analysts and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects. Job Title: Lead Data...


  • City Of London, United Kingdom Crimson Limited Full time

    This is a senior leadership role in data engineering at Crimson Limited, located in London.Key Responsibilities:Oversee and deliver large-scale data engineering projects that drive business outcomes.Develop and maintain scalable data platforms using technologies like Python, Spark, and Airflow.Design and implement APIs, machine learning models, and data...


  • London, Greater London, United Kingdom Data Science Festival Full time

    About the RoleAs a Staff Data Engineer at the Data Science Festival, you will play a key role in driving high standards across the data function. You will design, develop, and maintain data pipelines using Python and Azure, and collaborate with the wider data teams to achieve business objectives. The ideal candidate will have extensive experience with data...