Senior Data Pipeline Engineer

2 days ago


London, Greater London, United Kingdom Principle Full time
Job Description:

We are seeking a highly skilled Senior Data Pipeline Engineer to join our team at Principle. As a Lead Data Architect - SQL, you will be responsible for designing and building robust data pipelines that drive critical insights across the organisation.

Key Responsibilities:
  • Designing and optimising end-to-end data pipelines for efficient data flow and scalability
  • Building and maintaining Python-based ETL pipelines and data models for reliable data processing
  • Collaborating with cross-functional teams to support both batch and real-time data needs
  • Ensuring data accuracy, integrity, and performance across all data operations

Required Skills and Qualifications:
  • 5+ years of Data Engineering experience
  • Strong Python and SQL skills for data manipulation and pipeline development
  • Hands-on experience with Airflow (workflow orchestration) and Spark (large-scale data processing)
  • Ability to work with cross-functional teams

Benefits:
  • Competitive salary: £80,000 - £110,000 per annum
  • Bonus structure
  • Comprehensive benefits package

Others:

Candidates must be physically based in the UK, hold a valid Work Permit / Visa, and have the right to work in the UK without sponsorship.



  • London, Greater London, United Kingdom Northern Data Group Full time

    OverviewNorthern Data Group is a cutting-edge company that's changing the face of High-Performance Computing. With a strong focus on innovation and sustainability, we're shaping the future of data centers worldwide.Job SummaryWe're seeking a highly skilled Senior Data Engineer to join our team. As a key member of our Data Team, you will play a crucial role...


  • London, Greater London, United Kingdom Data Idols Full time

    Your Future Role:As a Staff Data Engineer at Data Idols, you will be responsible for designing, developing, and maintaining data pipelines that deliver high-quality insights. You will work closely with the wider data teams to ensure seamless data flow and implement best practices to drive continuous improvement in data quality and delivery. This is an...

  • Data Engineer

    7 days ago


    London, Greater London, United Kingdom Data Science Festival Full time

    Company OverviewThe Data Science Festival is one of the best-known retail brands in the UK, looking for a skilled Staff Data Engineer to support the delivery of data transformation during an exciting time within the company. As a Staff Data Engineer, you will collaborate with the wider data teams to build and maintain data pipelines.


  • London, Greater London, United Kingdom Northern Data Group Full time

    About the TeamNorthern Data Group is a leading provider of High-Performance Computing (HPC) solutions. Our team is dedicated to delivering innovative data-driven solutions to our clients.Main Responsibilities:Design, build, and maintain data pipelines and infrastructure to support data analysis and reporting.Collaborate with business stakeholders to...


  • London, Greater London, United Kingdom Data Idols Full time

    The OpportunityIn this key role, you will play a crucial part in driving high standards across the data function. You will support the team in building pipelines to deliver insights that enable informed decision-making, making us more data-driven. Additionally, you will work closely with the wider team, acting as a mentor to junior members and sharing your...


  • London, Greater London, United Kingdom Harnham - Data & Analytics Recruitment Full time

    £70,000 per annum, LondonAbout HarnhamWe are a dynamic and innovative insurance company with a passion for delivering high-quality solutions.Our team is committed to excellence and is now seeking a highly skilled Senior Python Pipeline Developer to join our team.In this role, you will be responsible for building and maintaining Python-based pipelines,...


  • London, Greater London, United Kingdom Harnham - Data & Analytics Recruitment Full time

    Job DescriptionWe are seeking a highly skilled Data Engineer to join our team in London. As part of our data migration project, you will be responsible for designing and managing ETL processes to support seamless data integration from new acquisitions.About the ProjectWe are working with a high-growth insurance brokerage to migrate their data to a modern...


  • London, Greater London, United Kingdom Data Idols Full time £85,000 - £95,000

    About the Role: Lead Data Pipeline DeveloperWe are seeking an experienced lead developer to join our data team as a Lead Data Pipeline Developer. In this role, you will be responsible for designing, developing, and maintaining complex data pipelines that deliver business-critical insights. The ideal candidate will have experience in data warehousing, data...


  • London, Greater London, United Kingdom Palantir Technologies Full time

    Palantir Technologies is looking for an Enterprise Data Pipeline Engineer to join our team. In this role, you will be responsible for designing and implementing large-scale data pipelines that meet the needs of our customers.The estimated salary for this position is $140,000 per year, based on the national average for a senior software engineer with similar...


  • London, Greater London, United Kingdom Data Idols Full time £75,000 - £100,000

    Company OverviewData Idols is a leading UK-based brand seeking an experienced Senior Data Engineer to support the delivery of data transformation during an exciting time within the company. With a strong focus on building and maintaining data pipelines using modern data technologies in a cloud environment, this role requires extensive experience with data...


  • London, Greater London, United Kingdom Opus Recruitment Solutions Full time

    Job DescriptionAs a Data Engineer, you will play a key role in shaping the data infrastructure and analytics capabilities of our client, a leading start-up in the financial sector. Your primary responsibilities will include:Developing and maintaining scalable and performant data pipelines using cloud infrastructure.Optimizing existing pipelines for speed,...


  • London, Greater London, United Kingdom Data Idols Full time £75,000 - £95,000

    **About Data Idols**Data Idols is a renowned retail brand in the UK, seeking an exceptional Senior Data Engineer to drive data transformation. As a Senior Data Engineer at Data Idols, you will play a pivotal role in delivering high-quality data solutions.The ideal candidate will have extensive experience with data pipelines, data warehousing, and cloud-based...


  • London, Greater London, United Kingdom Data Science Festival Full time

    About the RoleWe are looking for a talented Full-stack Data Engineer to join our team at Data Science Festival. In this role, you will be responsible for improving data structures, pipelines, and reporting.Key Responsibilities:Modernise data pipelines, data standards, and data reportsImprove data handling and development processesAs a Full-stack Engineer,...

  • Senior Data Architect

    4 weeks ago


    London, Greater London, United Kingdom Data Idols Full time

    Senior Data EngineerThe OpportunityWe are seeking a Senior Data Engineer to join our team at Data Idols, a leading retail brand in the UK. As a Senior Data Engineer, you will play a key role in driving high standards across the data function and building reliable and high-quality data solutions.Key ResponsibilitiesDesign and develop data pipelines using...


  • London, Greater London, United Kingdom ISR Recruitment Full time

    Senior Data EngineerISR Recruitment is seeking a highly skilled Senior Data Engineer to join our client's team on a major central government project.The Opportunity: This exciting role involves providing data services to develop robust data pipelines, analytics and visualisation tools that support the integration of multiple infrastructure datasets across...


  • London, Greater London, United Kingdom Data Idols Full time

    Drive Business Growth with Data-Driven InsightsAt Data Idols, we believe that data is the key to driving business growth. As a Senior Data Engineer, you will play a critical role in designing and implementing data pipelines that provide actionable insights to our stakeholders.Your Key ResponsibilitiesDesign and implement data pipelines using Python and...


  • London, Greater London, United Kingdom Xcede Full time

    About the RoleXcede is seeking an experienced Senior Image Data Pipeline Specialist to join our team in London. The successful candidate will be responsible for designing and implementing scalable data pipelines for advanced analytics and machine learning applications. The ideal candidate will have a strong background in data engineering, image data...


  • London, Greater London, United Kingdom FinTop Consulting Full time

    Job Summary: We are seeking a highly skilled Senior Data Engineer to join our team at FinTop Consulting. As a Senior Data Engineer, you will design and develop scalable data pipelines, automate data processes, and ensure data quality. This role requires strong experience as a Data Engineer with knowledge of the Azure stack, proficiency in Python, PySpark,...


  • London, Greater London, United Kingdom Data Idols Full time £75,000 - £95,000

    **Overview of Role**We are seeking a highly skilled Senior Data Engineer to join our team at Data Idols. As a key member of our data function, you will be responsible for designing, building, and maintaining robust data pipelines that drive business growth.The successful candidate will have a strong background in data engineering, with expertise in data...


  • London, Greater London, United Kingdom Vallum Associates Full time

    Senior Data Pipeline SpecialistWe are seeking an experienced Senior Data Pipeline Specialist to join our team at Vallum Associates. This role is critical in designing, implementing, and maintaining efficient data pipelines using Apache Kafka. As a key member of our engineering team, you will work closely with stakeholders to identify business needs and...