Lead Data Engineer

1 day ago

London Area, United Kingdom Cornwallis Elt Full time

Lead Data Engineer – Python, SQL, dbt, AWS, Dremio, Data Mesh, ETL, Data Pipelines, Data Warehousing, DWH, S3, Glue, Data Lake, Automation, London


A Lead Data Engineer is sought after by a leading workplace pensions provider, to run their expanding data engineering team in their London City office. The team have recently built out a next-gen data lake hosted in AWS which makes use of Dremio, Data Mesh architecture and a wide range of AWS tools (S3, Glue, Step Functions etc.).

With the infrastructure of this new Data Lake in place, there is now a focus on enhancing the data stored within (via monitoring, cleaning and, as well as a requirement to design, build and implement robust ETL pipelines for effective integration with the wider business.


As a Lead Engineer, you will also be responsible for overseeing a small team of Data Engineers, setting and driving best practises and standards, ensuring project delivery is kept on track and regularly engaging with stakeholders up to C-suite for feedback/updates.

This will also involve working directly alongside the Head of Data Platforms to ensure ongoing engineering work is aligned to overall architecture and roadmap, as well as future project planning.


If you demonstrate the following skillset, please do apply

  • 7/8 years’ experience in Python programming, particularly around data analysis, automation and the building of data pipelines (as well as an understanding of data warehousing concepts)
  • Strong SQL skills, especially in relation to ETL processes and data management (querying, cleaning, storing)
  • Experience working with and deploying to AWS cloud (having worked with any/all of S3, Glue, Lambda, Step Functions etc.)
  • Usage of PowerBI for data visualization
  • Experience of both Dremio and Data Mesh Architecture
  • Prior experience of leading software engineering teams (including line management)
  • Clear, confident communications with previous experience of working directly with business users

  • Data Engineering Delivery Lead

    4 weeks ago

    London, Greater London, United Kingdom NTT DATA Full time

    Lead Data Engineering Projects at a Global IT LeaderNTT DATA, a world-renowned IT service provider, is seeking a skilled and experienced Data Engineering Delivery Manager to join our dynamic team. As the 6th largest IT Service Provider globally, we are dedicated to driving business change and technology transformation across diverse industries.Your Role:In...

  • Data Engineering Delivery Lead

    2 weeks ago

    London, Greater London, United Kingdom NTT DATA Full time

    Lead Data Engineering Projects at a Global IT LeaderNTT DATA, a world-renowned IT service provider, is seeking a skilled and experienced Data Engineering Delivery Manager to join our dynamic team. As the 6th largest IT Service Provider globally, we are dedicated to driving business change and technology transformation across diverse industries.Your Role:In...

  • Data Engineering Lead

    1 week ago

    London, Greater London, United Kingdom Data Idols Full time £75,000 - £95,000

    Data Idols is seeking a Senior Data Engineer to support the delivery of data transformation during an exciting time within the company. As a Senior Data Engineer, you will collaborate with the wider data teams to build and maintain data pipelines. Your extensive experience with data pipelines, data warehousing, and driving high standards across the data...

  • Data Engineering Lead

    1 week ago

    London, Greater London, United Kingdom Data Idols Full time £75,000 - £95,000

    Data Idols is seeking a Senior Data Engineer to support the delivery of data transformation during an exciting time within the company. As a Senior Data Engineer, you will collaborate with the wider data teams to build and maintain data pipelines. Your extensive experience with data pipelines, data warehousing, and driving high standards across the data...

  • Data Engineering Practice Lead

    5 days ago

    London, Greater London, United Kingdom ON Data Staffing Full time

    Senior Engineering ManagerOur client, a leading Data & AI Consultancy, is seeking a seasoned professional to lead and grow their data engineering practice in the UK. As a Senior Consulting Manager in Data Engineering, you will be responsible for delivering cutting-edge data engineering solutions, building and expanding partner channels, and driving business...

  • Data Engineering Practice Lead

    5 days ago

    London, Greater London, United Kingdom ON Data Staffing Full time

    Senior Engineering ManagerOur client, a leading Data & AI Consultancy, is seeking a seasoned professional to lead and grow their data engineering practice in the UK. As a Senior Consulting Manager in Data Engineering, you will be responsible for delivering cutting-edge data engineering solutions, building and expanding partner channels, and driving business...

  • Data Engineering Practice Lead

    5 days ago

    London, Greater London, United Kingdom ON Data Staffing Full time

    Lead Data Engineering PracticeWe are seeking a seasoned Data Engineering leader to drive the growth of our Data Engineering practice in the UK. As a Senior Consulting Manager, you will be responsible for delivering cutting-edge data engineering solutions, building and expanding partner channels, and driving business growth.Key ResponsibilitiesLead and manage...

  • Data Engineering Practice Lead

    5 days ago

    London, Greater London, United Kingdom ON Data Staffing Full time

    Lead Data Engineering PracticeWe are seeking a seasoned Data Engineering leader to drive the growth of our Data Engineering practice in the UK. As a Senior Consulting Manager, you will be responsible for delivering cutting-edge data engineering solutions, building and expanding partner channels, and driving business growth.Key ResponsibilitiesLead and manage...

  • Technical Delivery Lead

    3 weeks ago

    London, Greater London, United Kingdom NTT DATA Full time

    About the RoleWe are seeking an experienced Senior Data Engineering Project Manager to join our team at NTT DATA. As a key member of our Data Engineering service line, you will be responsible for overseeing Data Engineering projects and programmes across our Data transformation portfolio.Key ResponsibilitiesLead highly-skilled technical teams in a...

  • Technical Delivery Lead

    2 weeks ago

    London, Greater London, United Kingdom NTT DATA Full time

    About the RoleWe are seeking an experienced Senior Data Engineering Project Manager to join our team at NTT DATA. As a key member of our Data Engineering service line, you will be responsible for overseeing Data Engineering projects and programmes across our Data transformation portfolio.Key ResponsibilitiesLead highly-skilled technical teams in a...

  • Lead Data Engineering Specialist

    2 weeks ago

    London, Greater London, United Kingdom ON Data Staffing Full time

    About the RoleWe are seeking a highly skilled Senior Data Solutions Architect to join our team at ON Data Staffing. As a key member of our data engineering team, you will play a pivotal role in delivering innovative big data solutions to our clients.Key ResponsibilitiesClient EngagementCollaborate directly with our clients to understand their business...

  • Lead Data Engineering Specialist

    3 weeks ago

    London, Greater London, United Kingdom ON Data Staffing Full time

    About the RoleWe are seeking a highly skilled Senior Data Solutions Architect to join our team at ON Data Staffing. As a key member of our data engineering team, you will play a pivotal role in delivering innovative big data solutions to our clients.Key ResponsibilitiesClient EngagementCollaborate directly with our clients to understand their business...

  • Data Architect Lead

    6 days ago

    London, Greater London, United Kingdom Data Idols Full time £80,000

    Data Idols is working with one of the world's largest gaming organisations to hire a Data Technical Lead. This is an opportunity to work at scale on leading the software architecture design as well as evolve and formulate functions. You will collaborate with wider teams to implement technical designs on regulatory reporting.**Key Responsibilities:*** Lead...

  • Data Architect Lead

    6 days ago

    London, Greater London, United Kingdom Data Idols Full time £80,000

    Data Idols is working with one of the world's largest gaming organisations to hire a Data Technical Lead. This is an opportunity to work at scale on leading the software architecture design as well as evolve and formulate functions. You will collaborate with wider teams to implement technical designs on regulatory reporting.**Key Responsibilities:*** Lead...

  • Lead Data Engineer

    1 month ago

    London Area, United Kingdom Dragonfly People Full time

    Lead Data EngineerData | Engineer | Lead | AWS | Python | SQL | Airflow | Snowflake | DBT | Fintech A Lead Data Engineer is required by a rapidly growing consumer Fin-Tech business who are based in London. The Lead Data Engineer will mentor and coach a team of Data Engineers, while driving automation and seeking opportunities to optimise data resources. You...

  • Lead Data Engineer

    1 month ago

    London Area, United Kingdom Dragonfly People Full time

    Lead Data Engineer Data | Engineer | Lead | AWS | Python | SQL | Airflow | Snowflake | DBT | Fintech A Lead Data Engineer is required by a rapidly growing consumer Fin-Tech business who are based in London. The Lead Data Engineer will mentor and coach a team of Data Engineers, while driving automation and seeking opportunities to optimise data resources....

  • Lead Data Engineer

    1 month ago

    London Area, United Kingdom Dragonfly People Full time

    Lead Data EngineerData | Engineer | Lead | AWS | Python | SQL | Airflow | Snowflake | DBT | Fintech A Lead Data Engineer is required by a rapidly growing consumer Fin-Tech business who are based in London. The Lead Data Engineer will mentor and coach a team of Data Engineers, while driving automation and seeking opportunities to optimise data resources. You...

  • Data Architect Lead

    1 week ago

    London, Greater London, United Kingdom Data Idols Full time £80,000

    Data Idols is working with one of the world's largest gaming organisations to hire a Data Technical Lead. This is an opportunity to work at scale on leading the software architecture design as well as evolve and formulate functions. You will collaborate with wider teams to implement technical designs on regulatory reporting.**Key Responsibilities:*** Lead...

  • Data Architect Lead

    1 week ago

    London, Greater London, United Kingdom Data Idols Full time £80,000

    Data Idols is working with one of the world's largest gaming organisations to hire a Data Technical Lead. This is an opportunity to work at scale on leading the software architecture design as well as evolve and formulate functions. You will collaborate with wider teams to implement technical designs on regulatory reporting.**Key Responsibilities:*** Lead...

  • Data Engineering Manager

    19 hours ago

    London, Greater London, United Kingdom Data Idols Full time £90,000 - £99,000

    Data Engineering Manager OpportunityData Idols is working with a leading retail brand in the UK to find a skilled Data Engineering Manager to support the delivery of data transformation during an exciting period of growth.The RoleAs a Data Engineering Manager, you will work closely with the teams to implement data strategy and deliver value throughout the...