Data Processing Pipeline Designer

5 days ago


Edinburgh, Edinburgh, United Kingdom Be-IT Resourcing Full time

We are looking for two experienced data engineers to join our team at Be-IT Resourcing on a 6-month contract. The ideal candidates will have a strong background in designing and building data processing pipelines using native AWS services.

About the Contract

The contract is for an initial 6 months, with the possibility of extension depending on performance and business needs.

Key Responsibilities
  • Design and build complex data systems to ingest and process large data volumes of data using native AWS services.
  • Develop and implement data processing pipelines using Pyspark, Python, SQL, and AWS Services such as Glue, Step Functions, and Redshift.
  • Maintain and improve existing data systems to ensure optimal performance and efficiency.

The estimated salary range for this role is £400 - 450 per day (outside IR35), with flexibility in terms of working hours and location.



  • Edinburgh, Edinburgh, United Kingdom Digital Preservation Coalition Full time

    Data Pipeline Specialist PositionEstimated salary: £55,000 - £75,000 per annum.National Records of Scotland is seeking a highly skilled Data Pipeline Specialist to join our team. As a Data Pipeline Specialist, you will be responsible for designing and implementing data pipelines using Microsoft SQL Server and SSIS.Job Description:You will work...


  • Edinburgh, Edinburgh, United Kingdom AI Connect Full time

    About the Role">AI Connect is seeking a talented Senior Data Engineer to join our team in Edinburgh or London. As a Senior Data Engineer, you will be responsible for developing innovative data-driven solutions, driving customer value, and promoting the adoption of new data techniques within our organization.">Key Responsibilities:">">Develop and implement...


  • Edinburgh, Edinburgh, United Kingdom Bright Purple Full time

    Cloud Data Architect - New Year OpportunitiesBright Purple's esteemed client is inviting applications from experienced Cloud Data Architects to join a high-profile greenfield project in the UK.The successful candidate will design and develop an end-to-end data pipeline using AWS services, ensuring scalability, reliability, and security.Design and...


  • Edinburgh, Edinburgh, United Kingdom Unitech Consultancy Full time

    Unitech Consultancy is expanding its data team and seeks a skilled Sr. Data Pipeline Developer to join our ranks.In this role, you will be responsible for designing, implementing, and maintaining complex data pipelines, leveraging cloud-based technologies like AWS and Snowflake.Job DescriptionDevelop and deploy data pipelines using Informatica PowerCenter,...


  • Edinburgh, Edinburgh, United Kingdom ADLIB Full time

    Data Engineer Job DescriptionWe are seeking a skilled Data Engineer to join our team at ADLIB, a leading UK-based financial services company.Main ResponsibilitiesThe successful candidate will be responsible for:Crafting and managing sophisticated data pipelines on cloud and on-prem platforms.Collaborating with stakeholders to leverage data in new, innovative...

  • Senior Engineer

    4 weeks ago


    Edinburgh, Edinburgh, United Kingdom oilandgas Full time

    Job DescriptionThis is an exciting opportunity to join our Civil Pipeline Engineering group as a Senior Engineer - Civil Pipeline. In this role, you will be responsible for managing complex projects, leading high-performing teams, and driving innovation in pipeline design. Your expertise in specialist pipeline design, including materials, jointing, and...

  • AWS Data Specialist

    1 week ago


    Edinburgh, Edinburgh, United Kingdom Edinburgh City Football Club Limited. Full time

    We are seeking a skilled Data Specialist to join our Business Intelligence & Data Analytics team at Edinburgh City Football Club Limited.Job OverviewThe ideal candidate will have experience working in the financial services sector and be proficient in designing and implementing ETL pipelines using Informatica, as well as cloud integration solutions using...


  • Edinburgh, Edinburgh, United Kingdom Nine Twenty Full time

    Job Description:As a Data Engineer at Incentive Games, you will play a crucial role in our data team. Your primary responsibility will be to design, implement, and maintain robust data infrastructure and pipelines, ensuring efficient data processing and availability for analysis and decision-making.This includes designing and developing scalable ETL...

  • Cloud Data Engineer

    4 weeks ago


    Edinburgh, Edinburgh, United Kingdom Bright Purple Full time

    About the RoleWe are seeking a highly skilled Cloud Data Engineer to join our team at Bright Purple. As a key member of our data engineering team, you will be responsible for designing and developing robust ETL data pipelines using Python and its related libraries.The ideal candidate will have 3+ years of commercial experience in data, machine learning, or...


  • Edinburgh, Edinburgh, United Kingdom oilandgas Full time

    Job OverviewWe are seeking a highly skilled Director of Pipeline Engineering to join our team at AtkinsRéalis. As a key member of our Civil Pipeline Engineering group, you will play a vital role in shaping the future of our pipeline service line. This is an exciting opportunity to work on iconic engineering projects both locally and internationally.In this...


  • Edinburgh, Edinburgh, United Kingdom Source Technology Full time

    Job Title: Data Engineer ExpertWe are seeking an accomplished Data Engineer to craft and maintain sophisticated data processing systems in the Financial Services industry.With a strong background in data engineering, you will be responsible for designing, developing, and implementing robust data pipelines and management systems. Your expertise will enable us...


  • Edinburgh, Edinburgh, United Kingdom AI Connect Full time

    Job Description">We are looking for a highly skilled Senior Data Engineer to join our team at AI Connect. In this role, you will be working with a major retail bank to develop innovative data-driven solutions.">Main Responsibilities:">">Design and implement data pipelines, models, and ETL processes to drive commercial success while maintaining data safety...


  • Edinburgh, Edinburgh, United Kingdom oilandgas Full time

    At AtkinsRéalis, we are seeking a highly skilled Civil Pipeline Engineering Director to join our team. This is an exciting opportunity to lead a diverse group of professionals and shape the future of our pipeline service line. The ideal candidate will have extensive experience in technical design, project management, and team leadership.The salary for this...


  • Edinburgh, Edinburgh, United Kingdom Nine Twenty Full time

    Responsibilities:Design and develop data infrastructure and pipelines to ensure efficient data processing and availability.Implement data quality checks, monitoring systems, and data governance practices to ensure data integrity.Collaborate with Data Scientists and Analysts to understand data requirements and deliver high-quality datasets.Develop and...


  • Edinburgh, Edinburgh, United Kingdom Apexon Full time

    About ApexonApexon is a pioneering digital-first technology services firm, dedicated to accelerating business transformation and delivering human-centric digital experiences. With a strong presence across four continents, our reputation is built on a comprehensive suite of engineering services, a commitment to solving clients' toughest technology problems,...


  • Edinburgh, Edinburgh, United Kingdom Trustpilot, Inc. Full time

    We are seeking a seasoned Data Governance Specialist to join our team at Trustpilot, Inc. in Edinburgh.The estimated salary for this role is £90,000 - £120,000 per year, depending on experience.Your primary responsibility will be to oversee the optimization of our data ecosystem's backbone, establishing key data governance processes and designing...

  • Data Scientist Lead

    6 days ago


    Edinburgh, Edinburgh, United Kingdom Build a Rocket Boy Full time

    Data Science Lead Job DescriptionWe are seeking a highly skilled Data Scientist Lead to join our team at Build a Rocket Boy. As a key member of our data science group, you will be responsible for leading the development of our data infrastructure and driving business growth through data-driven insights.The ideal candidate will have extensive experience in...


  • Edinburgh, Edinburgh, United Kingdom Source Technology Full time

    Job Overview:We're looking for an exceptional Data Engineer to join our team at Source Technology. As a key member of our data engineering team, you'll play a critical role in designing, building, and maintaining robust data pipelines and management systems.Key Responsibilities:Design and develop scalable data architectures to support business needs.Automate...


  • Edinburgh, Edinburgh, United Kingdom WorksHub Full time

    We're looking for a Data Architect and Engineer Leader at WorksHub who can design and implement scalable data systems.About the RoleThe successful candidate will have expertise in programming languages like Python, Java, and Scala, and a deep understanding of data processing and manipulation.ResponsibilitiesDesign and implement data pipelines to support...


  • Edinburgh, Edinburgh, United Kingdom Lane Clark & Peacock LLP. Full time

    LCP Delta is a specialist energy transition practice dedicated to supporting the energy sector in driving the transition. Our team is committed to furthering the transition across the whole value chain, focusing on power market modelling and downstream energy services markets.About the Role:We are seeking a skilled Junior/Mid level Data Engineer to join our...