Lead Data Engineer

4 weeks ago


London, United Kingdom Cornwallis Elt Ltd Full time
Job Description

Lead Data Engineer - Python, SQL, dbt, AWS, Dremio, Data Mesh, ETL, Data Pipelines, Data Warehousing, DWH, S3, Glue, Data Lake, Automation, London
A Lead Data Engineer is sought after by a leading workplace pensions provider, to run their expanding data engineering team in their London City office. The team have recently built out a next-gen data lake hosted in AWS which makes use of Dremio, Data Mesh architecture and a wide range of AWS tools (S3, Glue, Step Functions etc.).
With the infrastructure of this new Data Lake in place, there is now a focus on enhancing the data stored within (via monitoring, cleaning and as well as a requirement to design, build and implement robust ETL pipelines for effective integration with the wider business.
As a Lead Engineer, you will also be responsible for overseeing a small team of Data Engineers, setting and driving best practises and standards, ensuring project delivery is kept on track and regularly engaging with stakeholders up to C-suite for feedback/updates.
This will also involve working directly alongside the Head of Data Platforms to ensure ongoing engineering work is aligned to overall architecture and roadmap, as well as future project planning.
If you demonstrate the following skill set, please do apply
7/8 years' experience in Python programming, particularly around data analysis, automation and the building of data pipelines (as well as an understanding of data warehousing concepts)
Strong SQL skills, especially in relation to ETL processes and data management (querying, cleaning, storing)
Experience working with and deploying to AWS cloud (having worked with any/all of S3, Glue, Lambda, Step Functions etc.)
Usage of PowerBI for data visualization
Experience of both Dremio and Data Mesh Architecture
Prior experience of leading software engineering teams (including line management)
Clear, confident communications with previous experience of working directly with business users



  • London, United Kingdom Data Idols Full time €90,000 - €110,000

    Lead Data Engineer Location: London - Hybrid We are currently looking for a Lead Data Engineer to join our dynamic and forward-thinking team at a prominent British Insurance Company. This role is integral to our operations as it involves both maintenance of existing systems and the development of new projects. As a Lead Data Engineer, you will report...


  • London, United Kingdom Data Idols Full time €90,000 - €110,000

    Lead Data Engineer Location: London - Hybrid We are currently looking for a Lead Data Engineer to join our dynamic and forward-thinking team at a prominent British Insurance Company. This role is integral to our operations as it involves both maintenance of existing systems and the development of new projects. As a Lead Data Engineer, you will report...


  • London, United Kingdom Data Idols Full time

    Lead Data Engineer Location: London - Hybrid We are currently looking for a Lead Data Engineer to join our dynamic and forward-thinking team at a prominent British Insurance Company. This role is integral to our operations as it involves both maintenance of existing systems and the development of new projects. As a Lead Data Engineer, you will report...


  • London, Greater London, United Kingdom Data Idols Full time

    Data Idols is looking for a Chief Data Engineering Lead to support the delivery of data transformation.As a key member of the team, you will collaborate with the wider data teams and drive high standards across the data function. The role involves building pipelines to deliver insights enabling the company to be more data driven in their decision making.Key...


  • London, Greater London, United Kingdom Data Idols Full time

    The OpportunityWe are looking for a skilled Data Engineering Manager who has extensive experience in managing teams, implementing data strategy, and working with Azure. As a hands-off role, you will have the autonomy to lead the team and drive results.Your key responsibilities include:Implementing data strategy to drive business valueManaging and mentoring...


  • London, Greater London, United Kingdom Data Science Festival Full time

    We are seeking a highly skilled Data Engineering Manager to support the delivery of data transformation during an exciting time at Data Science Festival.As a key member of the Data Engineering team, you will work closely with cross-functional teams to implement data strategy and deliver value across the organization.The ideal candidate will have extensive...


  • London, Greater London, United Kingdom Data Science Festival Full time

    Data Engineering Manager Role OverviewAt the forefront of our company's innovative landscape, we seek a seasoned Data Engineering Manager to spearhead data transformation initiatives. With an exciting tenure at one of the UK's most renowned retail brands, this pivotal role requires expertise in implementing data strategy and delivering value across the...


  • London, Greater London, United Kingdom Data Science Festival Full time

    **Job Overview**Data Science Festival is seeking an experienced Big Data Engineering Lead to join their team in London – hybrid. This is a fantastic opportunity for a skilled professional to drive data transformation and deliver value throughout the company.This hands-off role requires someone with extensive experience in managing teams, stakeholder...


  • London, Greater London, United Kingdom Data Science Festival Full time

    Data Engineering Manager OpportunityWe are seeking an experienced Data Engineering Manager to lead our team in implementing data strategy and delivering value throughout the company.The RoleThe successful candidate will have extensive experience in managing teams, stakeholder management, and implementing data strategy. They will work closely with the teams...


  • London, Greater London, United Kingdom Data Intellect Full time

    Job SummaryWe are seeking a highly skilled Cloud Data Engineering Team Lead to spearhead our Databricks practice. This is a leadership role responsible for driving the growth, innovation, and successful delivery of cloud-based data solutions for our clients leveraging Databricks.Key ResponsibilitiesPractice LeadershipDevelop and lead the Databricks practice,...

  • Data Lead

    6 months ago


    London, United Kingdom Data Idols Full time

    **Data Science Lead**: Data Idols are working with a disrupter within the delivery service space. They are looking for a Data Science Lead to join their growing team and support them as they invest heavily in their data platform. **The Opportunity**: This opportunity comes with huge growth and the chance to really put your mark on this role. The company are...

  • Data Engineer

    1 month ago


    London, United Kingdom Data Idols Full time

    Data Engineer Salary: £400 - £500 a day Location: London, Hybrid Data Idols are working with a national software company that helps the public sector build better data solutions. They are looking for a Full-stack Data Engineer contractor to join them for an initial three months to assist in modernising data pipelines, data standards and data reports. ...

  • Data Engineer

    1 month ago


    London, United Kingdom Data Idols Full time

    Data Engineer Salary: £400 - £500 a dayLocation: London, HybridData Idols are working with a national software company that helps the public sector build better data solutions. They are looking for a Full-stack Data Engineer contractor to join them for an initial three months to assist in modernising data pipelines, data standards and data reports.The...

  • Data Engineer

    2 months ago


    London, United Kingdom Data Idols Full time

    Data Engineer Salary: £400 - £500 a dayLocation: London, HybridData Idols are working with a national software company that helps the public sector build better data solutions. They are looking for a Full-stack Data Engineer contractor to join them for an initial three months to assist in modernising data pipelines, data standards and data reports.The...

  • Data Engineer

    1 month ago


    London, United Kingdom Data Idols Full time

    Data Engineer Salary: £400 - £500 a dayLocation: London, HybridData Idols are working with a national software company that helps the public sector build better data solutions. They are looking for a Full-stack Data Engineer contractor to join them for an initial three months to assist in modernising data pipelines, data standards and data reports.The...

  • Data Engineer

    1 month ago


    London, United Kingdom Data Idols Full time

    Data Engineer Salary: £400 - £500 a dayLocation: London, HybridData Idols are working with a national software company that helps the public sector build better data solutions. They are looking for a Full-stack Data Engineer contractor to join them for an initial three months to assist in modernising data pipelines, data standards and data reports.The...

  • Data Engineer

    1 month ago


    London, United Kingdom Data Idols Full time €400 - €500

    Data Engineer Salary: £400 - £500 a day Location: London, Hybrid Data Idols are working with a national software company that helps the public sector build better data solutions. They are looking for a Full-stack Data Engineer contractor to join them for an initial three months to assist in modernising data pipelines, data standards and data reports. ...

  • Data Engineer

    1 month ago


    London, United Kingdom Data Idols Full time €400 - €500

    Data Engineer Salary: £400 - £500 a day Location: London, Hybrid Data Idols are working with a national software company that helps the public sector build better data solutions. They are looking for a Full-stack Data Engineer contractor to join them for an initial three months to assist in modernising data pipelines, data standards and data reports. ...


  • London, Greater London, United Kingdom Data Science Festival Full time

    About Our ClientOur client is a renowned software company dedicated to providing innovative data solutions to the public sector. With a strong focus on collaboration and customer satisfaction, they strive to make a positive impact on society.They are now seeking a talented Full-stack Data Engineer to join their team for an initial three-month contract. This...

  • Data Systems Engineer

    3 weeks ago


    London, Greater London, United Kingdom Data Privacy Full time

    About the Role:We are seeking a skilled Data Systems Engineer to join our team at Houseful.Key Responsibilities:Design and implement data warehousing solutions using AWS technologies.Collaborate with senior engineers to ensure data accuracy and implement validation checks.Develop and maintain ETL/ELT pipelines, ensuring alignment with business...