Lead Data Engineer

3 months ago


London, UK, United Kingdom Cornwallis Elt Full time
Lead Data Engineer – Python, SQL, dbt, AWS, Dremio, Data Mesh, ETL, Data Pipelines, Data Warehousing, DWH, S3, Glue, Data Lake, Automation, London

A Lead Data Engineer is sought after by a leading workplace pensions provider, to run their expanding data engineering team in their London City office. The team have recently built out a next-gen data lake hosted in AWS which makes use of Dremio, Data Mesh architecture and a wide range of AWS tools (S3, Glue, Step Functions etc.).
With the infrastructure of this new Data Lake in place, there is now a focus on enhancing the data stored within (via monitoring, cleaning and, as well as a requirement to design, build and implement robust ETL pipelines for effective integration with the wider business.

As a Lead Engineer, you will also be responsible for overseeing a small team of Data Engineers, setting and driving best practises and standards, ensuring project delivery is kept on track and regularly engaging with stakeholders up to C-suite for feedback/updates.
This will also involve working directly alongside the Head of Data Platforms to ensure ongoing engineering work is aligned to overall architecture and roadmap, as well as future project planning.

If you demonstrate the following skillset, please do apply
• 7/8 years’ experience in Python programming, particularly around data analysis, automation and the building of data pipelines (as well as an understanding of data warehousing concepts)
• Strong SQL skills, especially in relation to ETL processes and data management (querying, cleaning, storing)
• Experience working with and deploying to AWS cloud (having worked with any/all of S3, Glue, Lambda, Step Functions etc.)
• Usage of PowerBI for data visualization
• Experience of both Dremio and Data Mesh Architecture
• Prior experience of leading software engineering teams (including line management)
• Clear, confident communications with previous experience of working directly with business users

  • Lead Data Engineer

    3 months ago


    London,, UK, United Kingdom Cornwallis Elt Full time

    Lead Data Engineer – Python, SQL, dbt, AWS, Dremio, Data Mesh, ETL, Data Pipelines, Data Warehousing, DWH, S3, Glue, Data Lake, Automation, LondonA Lead Data Engineer is sought after by a leading workplace pensions provider, to run their expanding data engineering team in their London City office. The team have recently built out a next-gen data lake...

  • Lead Data Engineer

    2 months ago


    London,, UK, United Kingdom Mastek Full time

    Lead Data EngineerPermanent OpportunityLocation: London, UK – 2 days travel to officeOverview: We are seeking a highly skilled Lead Data Engineer to join our team for a critical project with NHS England. The project involves migrating Privacy Enhancement Technology (PET) from the existing Privitar system to IQVIA across multiple data platforms. The ideal...

  • Lead Data Engineer

    4 weeks ago


    London,, UK, United Kingdom JSS Search Full time

    JSS are partnering with a globally renowned, luxury brand who are looking for a Lead Data Engineer to join their growing and ambitious Data team. Reporting into the Head of Data and initially leading a small team of Data Engineers.My client is looking for a Data Engineering Lead with a breadth of experience to lead a group of data engineers in designing,...

  • Lead Data Engineer

    2 months ago


    London,, UK, United Kingdom Maclean Moore Full time

    Lead Data Engineer | Permanent | London | Hybrid | Financial ServicesI am partnered with a client of mine within the Financial Services Sector who are looking for an experienced Lead Data Engineer to manage their data infrastructure and lead their engineering team. This role requires expertise in designing, building, and maintaining scalable data pipelines...

  • Lead Data Engineer

    4 weeks ago


    London,, UK, United Kingdom Noir Full time

    Lead Data Engineer – Investment Banking – London/ Hybrid(Data Engineer, SQL Data Engineer, Java, Python, Spark, Scala, SQL, OO programming, Snowflake, Databricks, Data Fabric, design patterns, SOLID principles, ETL, Unit testing, NUnit, MSTest, Junit, Microservices Architecture, Continuous Integration, Azure DevOps, AWS, Jenkins, Agile, Data Engineer,...

  • Lead Data Engineer

    4 weeks ago


    London,, UK, United Kingdom Curated Full time

    Lead Data Engineer London - Hybrid 3 days pw in office.Gaming Tech Scale UpWe’re working with a dynamic and fast-growing global digital marketing technology specialising in innovative tech solutions within the iGaming sector. Having organically reached profitability, they're considering to grow consciously. In 2024 they made a number of key hires,...

  • Lead Data Engineer

    4 weeks ago


    London, UK, UK, United Kingdom Harnham Full time

    Lead Data EngineerLondon2 days a week in OfficeTHE COMPANYThis company is a well-known and highly regarded cosmetics company who are globally recognised as a luxury brand. Founded over 2 decades ago this company has been expanding and improving ever since. With the large number data they have accumulated they are looking to bring in a Lead Data Engineer to...

  • Lead Data Engineer

    4 weeks ago


    London,, UK, United Kingdom Core Consultants Full time

    Role Profile: Senior Data Engineering Lead – Internal Data Ingestion PlatformRole OverviewThe Senior Data Engineering Lead will spearhead the design, implementation, and technical oversight of an advanced internal data ingestion platform, enabling scalable and efficient ingestion of diverse datasets. The role involves leading a team of data engineers,...

  • Lead Data Engineer

    4 weeks ago


    London, UK, UK, United Kingdom iO Associates - UKEU Full time

    Lead Data Engineer - Up to £75,000 + Competitive benefits package - North West Office location - Flexible hybrid workingiO Associates is recruiting an experienced Lead Data Engineer on behalf of an exciting business in the eCommerce industry. This is a great time to join a growing Data team in a business that is undergoing significant digital transformation...

  • Lead Data Engineer

    4 weeks ago


    London,, UK, United Kingdom Understanding Recruitment Full time

    Lead Data Engineer - Up to £100k - HealthTech Fantastic new opportunity for an experienced Data Engineer to join a venture backed HealthTech startup.What You'll Be Doing: Architecting robust, secure data infrastructure from the ground upDesigning scalable data pipelines that can handle sensitive, complex informationCollaborating with multidisciplinary...

  • Lead Data Engineer

    2 weeks ago


    London,, UK, United Kingdom Finatal Full time

    Job Title: Lead Data EngineerContract: Initial 3 MonthsLocation: London (Hybrid)EM003Company Overview:Join a dynamic, Private Equity-backed IT Services Business experiencing rapid growth and innovation. We are embarking on a greenfield data project and seeking a hands-on Data Engineer to build and deliver a proof of concept for a cutting-edge data solution,...

  • Data Lead

    1 week ago


    London,, UK, United Kingdom adroc Full time

    We are looking for a Segmentation Lead Data Engineer for one of our public sector clients.Rate: up to £800 per dayDuration: until the 31st March 2025Inside IR35Location: LondonActive SC Clearance Highly desirableAs a Segmentation Lead Data Engineer you will:Excellent python, SQL and jinja skillsUnderstanding and awareness of RApplied automated testing, for...

  • Lead Data Engineer

    4 weeks ago


    London,, UK, United Kingdom Durlston Partners Full time

    Data Engineering Lead – Digital Assets | LondonAbout The CompanyJoin a leading global hedge fund with a dedicated digital assets division, committed to creating secure and efficient infrastructure for institutional-grade blockchain and cryptocurrency investments.About The RoleThey are seeking a Data Engineering Lead to spearhead the development of...


  • London,, UK, United Kingdom Dalton Full time

    Senior/Lead Data Engineer, Location: Hybrid London (2-3 days/week)About Dalton: Dalton is on a mission to make the world’s drug design more efficient. We are building the AI ecosystem for drug design and solving real-world problems that transform the efficiency of the pharmaceutical industry. Our mission is to harness cutting-edge technology and turn it...

  • Lead Data Engineer

    2 weeks ago


    London,, UK, United Kingdom First Derivative Full time

    Are you a technologist who has a hunger to dig deep and get involved in providing first-class solutions to real-world problems?Lead Data Engineer - LondonFirst Derivative is a people-first company. We are powered by our ever-growing teams of specialist problem-solvers and process-evolvers. Our extensive capabilities offer agile solutions and exciting career...

  • Controls Engineer

    1 month ago


    London,, UK, United Kingdom Vantage Data Centers Full time

    About Vantage Data CentersVantage Data Centers is a leading wholesale data center provider in North America with campuses in six strategic markets: Silicon Valley; Northern Virginia; Phoenix, Arizona; Quincy, Washington; Montreal and Quebec City, Canada. Founded in 2010, Vantage has an unceasing commitment to provide major hyperscale, cloud and enterprise...

  • Data Engineer

    4 weeks ago


    London,, UK, United Kingdom TRIA Full time

    Data Engineer / GCP / Digital£500 - £550 (Outside IR35)London – Hybrid 6 Month ContractMy client is a global ecommerce retailer based in London who are looking for a Data Engineer to join their dynamic data team to design, build and maintain their data pipelines in a cloud native environment. You’ll be working with the team on a data tool surrounding...

  • Senior Data Engineer

    2 months ago


    London,, UK, United Kingdom Dragonfly People Full time

    Senior/Lead Data EngineerA Senior/Lead Data Engineer is required by a rapidly growing Fintech based in London. The Senior/Lead Data Engineer will have extensive knowledge of modern technologies. This would ideally suit a Data Engineer with a passion for driving excellence and experience in a platform engineering or data engineering team. The role is 90%...

  • Lead Data Scientist

    2 months ago


    London,, UK, United Kingdom Harnham Full time

    LEAD DATA SCIENTIST - Data Science/MLREMOTE - 1 day per month in UxbridgeUp to £100,000 + car allowance + bonusCOMPANY:We are working with a market leading Telecommunications company with an established Data Science and AI team to bring a Lead Data Scientist into the team.They already have a strong established DS/ML team but are looking to grow this.They...

  • Data Engineer

    4 weeks ago


    London,, UK, United Kingdom Thyme Full time

    Data Science Engineer – AI-Powered Tech Startup - HybridJoin a trailblazing AI startup pushing boundaries in applied computer vision and real-time data processing. Recognized for innovation and backed by leading VC funds, they are building a next-generation solution that merges advanced tracking technology with exclusive, world-class content.As they scale,...