Lead Data Engineer

4 weeks ago

London Area, United Kingdom Inara Full time

Role: Lead Data Engineer

Location: London (Hybrid - 1 day per week in office)


Are you ready to design, implement, and manage secure, scalable data solutions in off-cloud environments?


We're looking for an experienced Lead Data Engineer to join our team and take the lead in optimising our data infrastructure. With a strong focus on PostgreSQL, PostGIS, and TimescaleDB, you will design APIs, manage data orchestration with AirFlow and SQLMesh, and prepare our systems for key security accreditations.


This role is ideal for someone passionate about data engineering and excited to work alongside technical leads to support our Data Sciences Roadmap.


What You'll Do:


  • Database Design & Optimisation: Architect and maintain highly efficient databases using PostgreSQL, PostGIS, and TimescaleDB, focusing on scalability for geospatial and time-series applications.
  • API Development: Design and build secure, scalable APIs to streamline data flow between frontend and backend systems.
  • Data Orchestration: Manage and optimise data pipelines using AirFlow and SQLMesh for smooth workflow automation.
  • Off-Cloud Deployment: Use Kubernetes (Talos, RKE2) and storage solutions like Ceph for deploying applications and databases in off-cloud settings.
  • Schema Management: Use Prisma for effective schema management, ensuring database efficiency and version control.
  • Security & Compliance: Drive system architecture towards accreditation standards by implementing best practices in data security, encryption, and role-based access control (RBAC).
  • Collaborate with the Principal Engineer: Work closely with our Principal Engineer on monitoring, DevSecOps, and system performance to ensure optimal integration and functionality.


What You Bring:


  • Extensive Expertise in PostgreSQL/PostGIS, TimescaleDB, and API design.
  • Off-Cloud Architecture Skills: Experience with Kubernetes (Talos, RKE2) and storage solutions like Ceph.
  • Data Orchestration: Proficiency with AirFlow and SQLMesh.
  • Strong Security & Compliance Knowledge: Experience with data encryption, RBAC, and secure systems design.
  • Performance Optimisation: Expertise in query optimisation, backup strategies, and clustering for high availability.
  • Monitoring & ETL: Experience with tools like Prometheus, Grafana, and database migration/ETL pipelines.


Bonus Skills:


  • Familiarity with NoSQL databases (e.g., MongoDB, Cassandra) and schema management tools like Prisma, Liquibase, or Flyway.
  • A knack for collaborating with DevOps teams to ensure system reliability and disaster recovery.


What We Offer:


  • Hybrid Work Model: Balance your time between our London hub and the comfort of your home.
  • Generous Leave: 25 days of leave + bank holidays (and your birthday)
  • Other Benefits: Including health insurance, access to a mental health support app, and equity options.
  • Exciting Challenges: Be a part of a team that’s preparing for future security certifications and authorisations for public and private sector organisations.


Important: Applicants must be UK residents with the right to work in the UK, as SC clearance may be required for this role in the near future.


Ready to make an impact as our Lead Data Engineer? Apply now and join us on this exciting journey

  • Data Engineering Delivery Lead

    2 months ago

    London, Greater London, United Kingdom NTT DATA Full time

    Lead Data Engineering Projects at a Global IT LeaderNTT DATA, a world-renowned IT service provider, is seeking a skilled and experienced Data Engineering Delivery Manager to join our dynamic team. As the 6th largest IT Service Provider globally, we are dedicated to driving business change and technology transformation across diverse industries.Your Role:In...

  • Data Engineering Delivery Lead

    2 months ago

    London, Greater London, United Kingdom NTT DATA Full time

    Lead Data Engineering Projects at a Global IT LeaderNTT DATA, a world-renowned IT service provider, is seeking a skilled and experienced Data Engineering Delivery Manager to join our dynamic team. As the 6th largest IT Service Provider globally, we are dedicated to driving business change and technology transformation across diverse industries.Your Role:In...

  • Data Engineer Consultant

    1 month ago

    London Area, United Kingdom she does data. Full time

    Job Title: Strategy & Data Consultant EngineerWe are seeking a highly skilled Data Consultant Engineer to join our team at She Does Data. As a key member of our team, you will play a crucial role in helping our clients achieve their business goals through data-driven insights.Key Responsibilities:Develop and implement data strategies to drive business growth...

  • Data Engineer Consultant

    1 month ago

    London Area, United Kingdom she does data. Full time

    Job Title: Strategy & Data Consultant EngineerWe are seeking a highly skilled Data Consultant Engineer to join our team at She Does Data. As a key member of our team, you will play a crucial role in helping our clients achieve their business goals through data-driven insights.Key Responsibilities:Develop and implement data strategies to drive business growth...

  • Technical Delivery Lead

    2 months ago

    London, Greater London, United Kingdom NTT DATA Full time

    About the RoleWe are seeking an experienced Senior Data Engineering Project Manager to join our team at NTT DATA. As a key member of our Data Engineering service line, you will be responsible for overseeing Data Engineering projects and programmes across our Data transformation portfolio.Key ResponsibilitiesLead highly-skilled technical teams in a...

  • Technical Delivery Lead

    2 months ago

    London, Greater London, United Kingdom NTT DATA Full time

    About the RoleWe are seeking an experienced Senior Data Engineering Project Manager to join our team at NTT DATA. As a key member of our Data Engineering service line, you will be responsible for overseeing Data Engineering projects and programmes across our Data transformation portfolio.Key ResponsibilitiesLead highly-skilled technical teams in a...

  • Senior Data Engineering Lead

    2 hours ago

    London, Greater London, United Kingdom Data Science Festival Full time

    We are seeking a highly skilled Data Engineering Manager to support the delivery of data transformation during an exciting time at Data Science Festival.As a key member of the Data Engineering team, you will work closely with cross-functional teams to implement data strategy and deliver value across the organization.The ideal candidate will have extensive...

  • Data Engineer

    3 weeks ago

    London, Greater London, United Kingdom Data Idols Full time

    Exciting Opportunity for a Full-Stack Data EngineerData Idols is working with a leading national software company that helps the public sector build better data solutions. We are seeking a skilled Full-Stack Data Engineer to join our team for an initial three months to assist in modernising data pipelines, data standards, and data reports.The OpportunityAs a...

  • Data Engineer

    3 weeks ago

    London, Greater London, United Kingdom Data Idols Full time

    Exciting Opportunity for a Full-Stack Data EngineerData Idols is working with a leading national software company that helps the public sector build better data solutions. We are seeking a skilled Full-Stack Data Engineer to join our team for an initial three months to assist in modernising data pipelines, data standards, and data reports.The OpportunityAs a...

  • Data Engineering Manager

    2 days ago

    London, Greater London, United Kingdom Data Science Festival Full time

    Data Engineering Manager OpportunityWe are seeking an experienced Data Engineering Manager to lead our team in implementing data strategy and delivering value throughout the company.The RoleThe successful candidate will have extensive experience in managing teams, stakeholder management, and implementing data strategy. They will work closely with the teams...

  • Lead Data Engineer

    3 weeks ago

    London Area, United Kingdom Xcede Full time

    Lead Data Engineer London (central) x3 days per week in officeUp to £100,000 Salary + Discretionary BonusOVERVIEWESG driven asset management company are hiring for a Lead Data Engineer to spearhead their Data Engineering team in London! Interesting and varied role where you can work across the whole pipeline on a breadth of impactful projects! Great scope...

  • Data Engineering Lead

    3 weeks ago

    London Area, United Kingdom Xcede Full time

    Data Engineering LeadLondon (central) x3 days per week in officeUp to £100,000 Salary + Discretionary BonusOVERVIEWESG driven asset management company are hiring for a Lead Data Engineer to spearhead their Data Engineering team in London! Interesting and varied role where you can work across the whole pipeline on a breadth of impactful projects! Great scope...

  • Lead Data Engineer

    3 weeks ago

    London Area, United Kingdom Xcede Full time

    Lead Data Engineer London (central) x3 days per week in officeUp to £100,000 Salary + Discretionary BonusOVERVIEWESG driven asset management company are hiring for a Lead Data Engineer to spearhead their Data Engineering team in London! Interesting and varied role where you can work across the whole pipeline on a breadth of impactful projects! Great scope...

  • Data Engineering Lead

    3 weeks ago

    London Area, United Kingdom Xcede Full time

    Data Engineering LeadLondon (central) x3 days per week in officeUp to £100,000 Salary + Discretionary BonusOVERVIEWESG driven asset management company are hiring for a Lead Data Engineer to spearhead their Data Engineering team in London! Interesting and varied role where you can work across the whole pipeline on a breadth of impactful projects! Great scope...

  • Lead Data Engineer

    2 weeks ago

    London Area, United Kingdom Xcede Full time

    Lead Data Engineer London (central) x3 days per week in office Up to £100,000 Salary + Discretionary Bonus OVERVIEW ESG driven asset management company are hiring for a Lead Data Engineer to spearhead their Data Engineering team in London! Interesting and varied role where you can work across the whole pipeline on a breadth of impactful projects! Great...

  • Data Engineering Lead

    3 weeks ago

    London Area, United Kingdom Xpertise Recruitment Full time

    Job Title: Lead Data EngineerXpertise Recruitment is seeking a highly skilled Lead Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for leading the design and development of scalable data pipelines and architecting robust data warehousing solutions.Key Responsibilities:Lead the data engineering function,...

  • Data Engineering Lead

    3 weeks ago

    London Area, United Kingdom Xpertise Recruitment Full time

    Job Title: Lead Data EngineerXpertise Recruitment is seeking a highly skilled Lead Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for leading the design and development of scalable data pipelines and architecting robust data warehousing solutions.Key Responsibilities:Lead the data engineering function,...

  • Cloud Data Engineering Team Lead

    2 days ago

    London, Greater London, United Kingdom Data Intellect Full time

    Job SummaryWe are seeking a highly skilled Cloud Data Engineering Team Lead to spearhead our Databricks practice. This is a leadership role responsible for driving the growth, innovation, and successful delivery of cloud-based data solutions for our clients leveraging Databricks.Key ResponsibilitiesPractice LeadershipDevelop and lead the Databricks practice,...

  • Data Engineering Manager

    2 months ago

    London, Greater London, United Kingdom Data Idols Full time £90,000 - £99,000

    Data Engineering Manager OpportunityData Idols is working with a leading retail brand in the UK to find a skilled Data Engineering Manager to support the delivery of data transformation during an exciting period of growth.The RoleAs a Data Engineering Manager, you will work closely with the teams to implement data strategy and deliver value throughout the...

  • Data Engineering Manager

    2 months ago

    London, Greater London, United Kingdom Data Idols Full time £90,000 - £99,000

    Data Engineering Manager OpportunityData Idols is working with a leading retail brand in the UK to find a skilled Data Engineering Manager to support the delivery of data transformation during an exciting period of growth.The RoleAs a Data Engineering Manager, you will work closely with the teams to implement data strategy and deliver value throughout the...