Lead Data Engineer

2 months ago


London, UK, United Kingdom Cornwallis Elt Full time

Lead Data Engineer – Python, SQL, dbt, AWS, Dremio, Data Mesh, ETL, Data Pipelines, Data Warehousing, DWH, S3, Glue, Data Lake, Automation, London


A Lead Data Engineer is sought after by a leading workplace pensions provider, to run their expanding data engineering team in their London City office. The team have recently built out a next-gen data lake hosted in AWS which makes use of Dremio, Data Mesh architecture and a wide range of AWS tools (S3, Glue, Step Functions etc.).

With the infrastructure of this new Data Lake in place, there is now a focus on enhancing the data stored within (via monitoring, cleaning and, as well as a requirement to design, build and implement robust ETL pipelines for effective integration with the wider business.


As a Lead Engineer, you will also be responsible for overseeing a small team of Data Engineers, setting and driving best practises and standards, ensuring project delivery is kept on track and regularly engaging with stakeholders up to C-suite for feedback/updates.

This will also involve working directly alongside the Head of Data Platforms to ensure ongoing engineering work is aligned to overall architecture and roadmap, as well as future project planning.


If you demonstrate the following skillset, please do apply

  • 7/8 years’ experience in Python programming, particularly around data analysis, automation and the building of data pipelines (as well as an understanding of data warehousing concepts)
  • Strong SQL skills, especially in relation to ETL processes and data management (querying, cleaning, storing)
  • Experience working with and deploying to AWS cloud (having worked with any/all of S3, Glue, Lambda, Step Functions etc.)
  • Usage of PowerBI for data visualization
  • Experience of both Dremio and Data Mesh Architecture
  • Prior experience of leading software engineering teams (including line management)
  • Clear, confident communications with previous experience of working directly with business users

  • Lead Data Engineer

    1 month ago


    London,, UK, United Kingdom Mastek Full time

    Lead Data EngineerPermanent OpportunityLocation: London, UK – 2 days travel to officeOverview: We are seeking a highly skilled Lead Data Engineer to join our team for a critical project with NHS England. The project involves migrating Privacy Enhancement Technology (PET) from the existing Privitar system to IQVIA across multiple data platforms. The ideal...

  • Lead Data Engineer

    6 days ago


    London,, UK, United Kingdom JSS Search Full time

    JSS are partnering with a globally renowned, luxury brand who are looking for a Lead Data Engineer to join their growing and ambitious Data team. Reporting into the Head of Data and initially leading a small team of Data Engineers.My client is looking for a Data Engineering Lead with a breadth of experience to lead a group of data engineers in designing,...

  • Lead Data Engineer

    3 weeks ago


    London,, UK, United Kingdom Maclean Moore Full time

    Lead Data Engineer | Permanent | London | Hybrid | Financial ServicesI am partnered with a client of mine within the Financial Services Sector who are looking for an experienced Lead Data Engineer to manage their data infrastructure and lead their engineering team. This role requires expertise in designing, building, and maintaining scalable data pipelines...

  • Lead Data Engineer

    1 month ago


    London,, UK, United Kingdom Understanding Recruitment Full time

    Data Engineer | West London (Hybrid Working)First 3 months Full Time then 2/3x per week there on in officeUp to £100,000 per annum doeI am looking for a Lead Data Engineer to join a cutting-edge tech team that are revolutionising the real estate industry through cloud tech!You will be joining an innovative and collaborative team based in Chiswick (West...

  • Lead Data Engineer

    1 month ago


    London,, UK, United Kingdom Innova Recruitment Full time

    We're working with an innovative client at the forefront of global trade, helping them find a passionate Lead Data Engineer to drive data innovation and strategy within their organisation. This is your opportunity to lead a team and build impactful data solutions in a cloud-based SaaS environment.The role requires a data engineering leader who can scale...

  • Lead Data Engineer

    6 days ago


    London,, UK, United Kingdom Noir Full time

    Lead Data Engineer – Investment Banking – London/ Hybrid(Data Engineer, SQL Data Engineer, Java, Python, Spark, Scala, SQL, OO programming, Snowflake, Databricks, Data Fabric, design patterns, SOLID principles, ETL, Unit testing, NUnit, MSTest, Junit, Microservices Architecture, Continuous Integration, Azure DevOps, AWS, Jenkins, Agile, Data Engineer,...

  • Lead Data Engineer

    6 days ago


    London,, UK, United Kingdom Curated Full time

    Lead Data Engineer London - Hybrid 3 days pw in office.Gaming Tech Scale UpWe’re working with a dynamic and fast-growing global digital marketing technology specialising in innovative tech solutions within the iGaming sector. Having organically reached profitability, they're considering to grow consciously. In 2024 they made a number of key hires,...

  • Lead Data Engineer

    6 days ago


    London, UK, UK, United Kingdom Harnham Full time

    Lead Data EngineerLondon2 days a week in OfficeTHE COMPANYThis company is a well-known and highly regarded cosmetics company who are globally recognised as a luxury brand. Founded over 2 decades ago this company has been expanding and improving ever since. With the large number data they have accumulated they are looking to bring in a Lead Data Engineer to...

  • Lead Data Engineer

    1 month ago


    London, UK, UK, United Kingdom Harnham Full time

    Lead Data Engineer London - Hybrid (3 days per week in office) Up to £90,000 + BenefitsOVERVIEWWe are partnering with a prominent investment company within the infrastructure sector, seeking a skilled Senior Data Engineer to help establish a data strategy and support long-term data-driven growth. With recent consulting insights highlighting the need for a...

  • Lead Data Engineer

    6 days ago


    London,, UK, United Kingdom Core Consultants Full time

    Role Profile: Senior Data Engineering Lead – Internal Data Ingestion PlatformRole OverviewThe Senior Data Engineering Lead will spearhead the design, implementation, and technical oversight of an advanced internal data ingestion platform, enabling scalable and efficient ingestion of diverse datasets. The role involves leading a team of data engineers,...

  • Lead Data Engineer

    6 days ago


    London, UK, UK, United Kingdom iO Associates - UKEU Full time

    Lead Data Engineer - Up to £75,000 + Competitive benefits package - North West Office location - Flexible hybrid workingiO Associates is recruiting an experienced Lead Data Engineer on behalf of an exciting business in the eCommerce industry. This is a great time to join a growing Data team in a business that is undergoing significant digital transformation...

  • Lead Data Engineer

    6 days ago


    London,, UK, United Kingdom Understanding Recruitment Full time

    Lead Data Engineer - Up to £100k - HealthTech Fantastic new opportunity for an experienced Data Engineer to join a venture backed HealthTech startup.What You'll Be Doing: Architecting robust, secure data infrastructure from the ground upDesigning scalable data pipelines that can handle sensitive, complex informationCollaborating with multidisciplinary...

  • Lead Data Engineer

    4 days ago


    London,, UK, United Kingdom Durlston Partners Full time

    Data Engineering Lead – Digital Assets | LondonAbout The CompanyJoin a leading global hedge fund with a dedicated digital assets division, committed to creating secure and efficient infrastructure for institutional-grade blockchain and cryptocurrency investments.About The RoleThey are seeking a Data Engineering Lead to spearhead the development of...

  • Lead Data Engineer

    1 month ago


    London,, UK, United Kingdom MarkJames Search Full time

    We are looking for a Lead Data Engineer to take charge of designing and implementing complex data flows across operational systems, analytics, and BI systems. As the lead, you will plan and drive the development of data solutions, ensuring a perfect balance between functional and non-functional requirements. This role also involves monitoring compliance with...

  • Data Engineering Lead

    4 weeks ago


    London-London, UK, United Kingdom myGwork Full time

    This job is with Mars, an inclusive employer and a member of myGwork – the largest global platform for the LGBTQ+ business community. Please do not contact the recruiter directly. Job Description:Are you passionate about Data and Analytics (D&A) and excited about how it can completely transform the way an enterprise works? Do you have the...


  • London,, UK, United Kingdom Dalton Full time

    Senior/Lead Data Engineer, Location: Hybrid London (2-3 days/week)About Dalton: Dalton is on a mission to make the world’s drug design more efficient. We are building the AI ecosystem for drug design and solving real-world problems that transform the efficiency of the pharmaceutical industry. Our mission is to harness cutting-edge technology and turn it...

  • Data Engineering Lead

    1 month ago


    London, UK, UK, United Kingdom Datatech Analytics Full time

    Data Engineering Lead – AWS & SnowflakeHybrid working: 3 days in TW6, Middlesex offices & 2 days homer/remoteSalary: Negotiable to £70,000 DOE plus 40 % bonus potentialJob Reference: J12869Full UK working rights required/no sponsorship availableTHE ROLELooking for a challenge in one of the world’s largest airfreight logistics organisation and a FTSE 100...

  • Controls Engineer

    3 weeks ago


    London,, UK, United Kingdom Vantage Data Centers Full time

    About Vantage Data CentersVantage Data Centers is a leading wholesale data center provider in North America with campuses in six strategic markets: Silicon Valley; Northern Virginia; Phoenix, Arizona; Quincy, Washington; Montreal and Quebec City, Canada. Founded in 2010, Vantage has an unceasing commitment to provide major hyperscale, cloud and enterprise...

  • Lead Data Scientist

    1 month ago


    London,, UK, United Kingdom Harnham Full time

    LEAD DATA SCIENTIST - Data Science/MLREMOTE - 1 day per month in UxbridgeUp to £100,000 + car allowance + bonusCOMPANY:We are working with a market leading Telecommunications company with an established Data Science and AI team to bring a Lead Data Scientist into the team.They already have a strong established DS/ML team but are looking to grow this.They...

  • Data Engineer

    6 days ago


    London,, UK, United Kingdom TRIA Full time

    Data Engineer / GCP / Digital£500 - £550 (Outside IR35)London – Hybrid 6 Month ContractMy client is a global ecommerce retailer based in London who are looking for a Data Engineer to join their dynamic data team to design, build and maintain their data pipelines in a cloud native environment. You’ll be working with the team on a data tool surrounding...