Infrastructure Data Engineer Lead

20 hours ago


London, Greater London, United Kingdom Opus Recruitment Solutions Full time
Opus Recruitment Solutions

We are looking for a highly skilled Infrastructure Data Engineer Lead to join our team in Bournemouth. This role offers the opportunity to work with industry-leading technology and develop your career in a dynamic environment.

About the Role:

  • Design, implement, and maintain scalable AWS infrastructure using Infrastructure as Code principles.
  • Develop and optimize data pipelines using technologies such as Spark, Python, and Java.
  • Implement and manage Iceberg tables for efficient data storage and retrieval.
  • Enforce data retention policies and ensure compliance with governance standards.
  • Collaborate with software engineers to align infrastructure with application needs.
  • Implement robust data security measures.
  • Monitor and troubleshoot data pipelines and infrastructure.
  • Maintain comprehensive documentation of infrastructure and processes.

Key Responsibilities:

  1. Design and implement scalable cloud infrastructure.
  2. Develop and optimize data pipelines.
  3. Implement and manage Iceberg tables.
  4. Enforce data retention policies.
  5. Collaborate with software engineers.

Requirements:

  • Experience in AWS Infrastructure and/or SRE roles.
  • Strong background in IaC using tools like Terraform or CloudFormation.
  • Proficiency in Spark, Java, and Python.
  • Experience with data orchestration tools (e.g., Airflow, Prefect, Dagster).
  • Knowledge of Postgres, GraphQL, and other data manipulation tools.
  • Familiarity with big data concepts, including Apache Iceberg and data lakes.
  • Experience with geospatial data formats (Parquet/GeoParquet, GeoJSON, Shapefiles).
  • Proficiency in DevOps tools (Git, Docker, Jenkins).
  • Understanding of networking concepts (IP Protocol, DNS, load balancing).
  • Excellent problem-solving and communication skills.

Salary: £65,000 - £85,000 per annum (dependent on experience).



  • London, Greater London, United Kingdom Data Science Festival Full time

    At Data Science Festival, we're seeking a highly skilled Senior Data Platform Engineer to shape the data landscape and drive digital transformation. As a key member of our team, you'll play a pivotal role in designing, implementing, and maintaining scalable data infrastructure using cutting-edge cloud-based technologies.Key ResponsibilitiesDesign and...


  • London, Greater London, United Kingdom Data Idols Full time

    About the RoleData Idols is a leading organisation in the field of data-driven decision-making. We are currently seeking a Senior Data Platform Engineer to join our team and contribute to shaping the data landscape. In this role, you will work alongside a talented, collaborative team to build, optimise, and secure scalable data infrastructure.Key...


  • London, Greater London, United Kingdom Data Idols Full time £90,000 - £110,000

    Job DescriptionWe are currently looking for a Lead Data Engineer to join our team. As a key member of our data engineering function, you will be responsible for designing, building, and maintaining robust ETL processes and data infrastructure. You will also write custom code to enhance data extraction, transformation, and loading processes.


  • London, Greater London, United Kingdom Data Idols Full time £80,000 - £90,000

    About the RoleWe're looking for a skilled Senior Data Platform Engineer to join our team and help shape the future of our data platform. As a key member of our data engineering team, you'll have the opportunity to work on a wide range of projects and contribute directly to data-driven decision-making.Key ResponsibilitiesDesign and implement scalable,...


  • London, Greater London, United Kingdom Northern Data Group Full time

    Job OverviewNorthern Data Group is seeking a Senior Data Engineer to join our team and play a crucial role in designing, building, and maintaining our data infrastructure.Estimated Salary: $120,000 - $180,000 per yearKey Responsibilities:Data Pipeline Development: Develop and maintain end-to-end data pipelines, ensuring performance, scalability, and...


  • London, Greater London, United Kingdom Understanding Recruitment Full time £100,000

    **Job Title:** Lead Data Infrastructure Engineer**Location:** London, with 1/2 day flexible working**Salary:** Up to £100,000 per annumJoin our team at Understanding Recruitment as a Lead Data Infrastructure Engineer and take on a challenging role where you will design and implement robust, secure data infrastructure from the ground up.You will work closely...


  • London, Greater London, United Kingdom Data Idols Full time £80,000 - £90,000

    About the RoleWe are currently seeking a Senior Data Platform Engineer to join our team at Data Idols, a renowned retailer in the digital-first era. This is an exciting opportunity to contribute to the design, implementation, and maintenance of a robust data platform essential to our ongoing digital transformation.Your Key Responsibilities:Design and...


  • London, Greater London, United Kingdom Opus Recruitment Solutions Full time

    We are looking for a Data Infrastructure Lead to join our Opus Recruitment Solutions team in London. As a Senior Data Engineer, you will be responsible for establishing the overall infrastructure and architecture of our client's Crypto space.The successful candidate will have strong experience with Databricks and Python, as well as Spark and PySpark. You...


  • London, Greater London, United Kingdom Opus Recruitment Solutions Full time

    Job Description:We are seeking a highly skilled Infrastructure and Data Engineering Lead to join our team at Opus Recruitment Solutions. This role offers the opportunity to work with cutting-edge cloud technologies, designing and implementing scalable infrastructure solutions using Infrastructure as Code principles.About the Role:Design, implement, and...


  • London, Greater London, United Kingdom Understanding Recruitment Full time £100,000

    We are looking for an experienced Infrastructure Engineer to join our team as a Lead Data Engineer.In this role, you will be responsible for:Designing and implementing secure, scalable data pipelinesCollaborating with cross-functional teams to deliver high-quality solutionsMaintaining and optimizing existing infrastructure to ensure optimal performanceTo...


  • London, Greater London, United Kingdom Data Idols Full time

    About Us">Data Idols is a leading insurance company that uses data-driven solutions to transform the industry. We are currently looking for a Lead Data Engineer to join our team and contribute to our success.The ideal candidate will have strong expertise in SQL and Python, as well as experience working with AWS cloud services. They should also have a solid...


  • London, Greater London, United Kingdom Data Careers Full time

    Job Description: Data Engineer SC Cleared We are looking to recruit a Data Engineer with SC Clearance for a leading software and solutions organisation. The successful candidate will design, develop and maintain robust data infrastructure and ETL pipelines that support the data needs of projects, services and customers. Key Responsibilities: Build data...


  • London, Greater London, United Kingdom KDR Talent Solutions Full time

    Data Infrastructure Lead Job Description:We are seeking an experienced Data Infrastructure Lead to join our team at KDR Talent Solutions. As a Data Infrastructure Lead, you will be responsible for designing, implementing, and maintaining our data infrastructure, including our legacy data warehouses and cloud-based systems.This is a unique opportunity to work...


  • London, Greater London, United Kingdom AlphaSights Full time

    Job Summary:We're looking for a talented Senior Data Engineer to lead our data infrastructure efforts in London. As a key member of our Engineering team, you'll design, develop, and deploy scalable data pipelines and architectures, ensuring high-performance data operations.Main Responsibilities:Data Pipeline Architecture: Design and implement robust data...


  • London, Greater London, United Kingdom Data Idols Full time £80,000 - £90,000

    Job DescriptionWe are currently seeking a Senior Data Platform Engineer to join our team. This role involves designing and implementing scalable, reliable, and secure data platform infrastructure using Azure Services.Maintaining an optimised platform by monitoring performance across compute and storage solutionsLeading CI/CD deployments for data...


  • London, Greater London, United Kingdom Data Science Festival Full time

    Job OverviewSenior Data Platform EngineerAs a Senior Data Platform Engineer at our prestigious retailer, you'll be part of a talented, collaborative team that's evolving into a digital-first, data-led organisation. Your expertise will shape the data landscape and ensure the delivery of high-quality data solutions.You'll work closely with cross-functional...


  • London, Greater London, United Kingdom Data Science Festival Full time

    About Our ClientOur client is a renowned software company dedicated to providing innovative data solutions to the public sector. With a strong focus on collaboration and customer satisfaction, they strive to make a positive impact on society.They are now seeking a talented Full-stack Data Engineer to join their team for an initial three-month contract. This...


  • London, Greater London, United Kingdom Northern Data Group Full time

    About the RoleWe are excited to announce a new opportunity for a Senior Data Infrastructure Developer to join our team at Northern Data Group. In this role, you will play a critical part in designing, building, and maintaining our data infrastructure.Main Responsibilities:Develop and maintain end-to-end data pipelines, focusing on performance, scalability,...


  • London, Greater London, United Kingdom Data Idols Full time

    Senior Data Platform EngineerWe are seeking a highly skilled Data Engineer to join our team at Data Idols. As a key member of our data platforms group, you will play a vital role in shaping the data landscape and contribute to data-driven decision-making.The OpportunityIn this exciting role, you will be responsible for designing, implementing, and...


  • London, Greater London, United Kingdom Data Science Festival Full time

    Are you a skilled professional looking to shape the data landscape? We are currently seeking a Senior Data Infrastructure Specialist to join our prestigious retailer's evolving data-led organisation.About the CompanyOur company is a leading player in the digital transformation space, leveraging cutting-edge cloud-based technologies to deliver high-quality...