Data Engineer

4 weeks ago


London, UK, United Kingdom Prism Digital Full time

Lead Azure Data Engineer | PySpark (Python) & Synapse | Tech for Good / Charity


Essential skills required:

  • Azure – solid experience required of the Azure Data ecosystem
  • Python - ESSENTIAL as PySpark is used heavily. You will be tested on PySpark
  • Azure Synapse - ESSENTIAL as is used heavily
  • Spark
  • Azure Data Lake / Data Bricks / Data Factory
  • Be happy to act as a lead and mentor to the other permanent Azure Data Engineers


This is the chance to work for a unique organisation Our client is a not-for-profit charity that reviews/certifies companies, governments and financial institutions for their environmental impact and awareness. Our client is the main global authority that certifies and monitors the environmental impact that companies/governments have on the natural environment. Essentially, they provide the equivalent of the “Fair Trade” sticker that you would see on a jar of coffee but for businesses and local governments.


The organisation is over 20 years old, headquartered in the UK with an office in central London and has 700 staff globally (100 circa in the UK).


You will be reporting to the CTO who manages the tech team of circa 30 people comprising developers, data engineers, QA and DevOps.


Essential skills required:

  • Azure – solid experience required of the Azure Data ecosystem
  • Python - ESSENTIAL as PySpark is used heavily. You will be tested on PySpark
  • Azure Synapse - ESSENTIAL as is used heavily
  • Spark
  • Azure Data Lake / Data Bricks / Data Factory
  • Be happy to act as a lead and mentor to the other permanent Azure Data Engineers


The tech stack you will be working on is:

  • Azure – a very solid Azure cloud native experience is essential
  • Data platforms on Azure; Synapse, Spark, Azure Data Factory
  • Python
  • Data design patterns, such as medallion architecture & data vault
  • SQL queries, views, stored procs, CTE etc – strong experience required
  • Terraform used for IaC
  • Azure DevOps used for CI/CD
  • DP-203 certification preferred but not essential


Specifics you will be working on in this contract:

  • Building data lakes

Implement medallion design

Created data standardisation and cleansing pipelines across zones/layers

Familiar with partitioning and efficient storage patterns

Work with high-volume pipelines in batch or streaming mode

  • Data Engineering

Configure Azure Synapse components such as integration runtimes, spark clusters and dedicated pools

Work with delta tables and Lakehouse architecture

Familiar with Synapse CI/CD

  • Azure Purview or another governance tool experience. Familiar with building Catalogs and lineage


Lead Azure Data Engineer | PySpark (Python), Synapse, Data Lake | Tech for Good / Charity


  • Senior Data Engineer

    2 weeks ago


    London, UK, UK, United Kingdom Data Idols Full time

    Senior Data EngineerSalary: up to £85,000Location: London (Hybrid)Data Idols are working with an exciting scale-up revolutionising delivery logistics and moving items across the UK with efficiency and ease all whilst cutting emissions. Trusted by millions of customers already they are expanding their data team to manage increased volumes of data and demand....


  • London, UK, United Kingdom Paritas Recruitment - Data & Tech Full time

    We are seeking a Data Architect to take the lead in a newly formed department within a trading house. Leading a multi disciplinary team, we are seeking individuals who can come in and make the decisions on future architecture designs, contribute to a wider data strategy and face off to a range of senior stakeholders across the business. Experience in other...

  • Data Engineer

    7 days ago


    London,, UK, United Kingdom The People Network Full time

    Data EngineerA Data Engineer is required to join a global consultancy providing top quality Data solutions to the Government sector.In this exciting Data Engineer role, you will work on designing, constructing, and maintaining scalable data pipelines and transforming raw data.This is a fantastic opportunity for someone who comes from a Data Engineering...

  • Data Engineer

    2 days ago


    London,, UK, United Kingdom W Talent Full time

    Are you a Data Engineer with a demonstrable background of building and maintaining data pipelines in Azure? There is a high growth technology consulting business we are working with at the moment to help find people like you to add to their highly skilled team. If you are an Azure Data Engineer looking to apply your skills on interesting projects and learn...

  • Data Engineer

    2 weeks ago


    London,, UK, United Kingdom Mastek Full time

    Job Description: Data EngineerLocation: [UK]SC Cleared: RequiredJob Type: Full-TimeExperience: 12-14 yearsJob description:We are seeking a highly skilled and motivated Data Engineer to join our dynamic team. The Data Engineer will be responsible for designing, building, and maintaining scalable data pipelines and infrastructure to support data-driven...

  • Data Engineer

    2 days ago


    London,, UK, United Kingdom Oliver Bernard Full time

    Data Engineer - up to £55k We have partnered with a growing SaaS company who operate within the hospitality sector. Their purpose is to boost performance by pulling together data from all different technologies within the business in real time to improve performance. As a Data Engineer you will be expected to build and maintain their data pipeline and ETL...

  • Data Engineer

    2 days ago


    London,, UK, United Kingdom Oliver Bernard Full time

    Data Engineer - up to £70kWe have partnered with a growing SaaS company who operate within the hospitality sector. Their purpose is to boost performance by pulling together data from all different technologies within the business in real time to improve performance.As a Data Engineer you will be expected to build and maintain their data pipeline and ETL...

  • Data Engineer

    1 month ago


    London,, UK, United Kingdom Response Informatics Full time

    We are looking for a Lead Data Engineer to join our Data Engineering Team. In this senior role, you will be responsible for managing our Enterprise Data Warehouse and ensuring the efficient flow and quality of data.Your tasks will include implementing data processes, managing data architecture, designing ETL processes, and analyzing data for business...

  • Data Engineer

    1 month ago


    London,, UK, United Kingdom Travelex Full time

    Job Title: Data EngineerJob Type: Full Time, PermanentWorking location: London, HybridRole Purpose At Travelex we are developing modern data services, which will be at the heart of our relationship with our customers. Our data architecture is becoming more service-oriented and more real-time – which creates a range of engineering problems for you to solve....

  • Data Engineer

    4 weeks ago


    London,, UK, United Kingdom Travelex Full time

    Job Title: Data EngineerJob Type: Full Time, PermanentWorking location: London, HybridRole Purpose At Travelex we are developing modern data services, which will be at the heart of our relationship with our customers. Our data architecture is becoming more service-oriented and more real-time – which creates a range of engineering problems for you to solve....

  • Data Engineer

    4 weeks ago


    London,, UK, United Kingdom Understanding Recruitment Full time

    Data EngineerDo you want to join a leading investment firm specializing in quantitative trading, driven by innovation and excellence? Keep reading! Location: London, UKAs a Data Engineer, you'll build and maintain infrastructure for trading strategies, collaborating with data scientists and analysts.Responsibilities: Data Pipelines Data Integration️...

  • Data Engineer

    4 weeks ago


    London,, UK, United Kingdom Understanding Recruitment Full time

    Data EngineerDo you want to join a leading investment firm specializing in quantitative trading, driven by innovation and excellence? Keep reading! Location: London, UKAs a Data Engineer, you'll build and maintain infrastructure for trading strategies, collaborating with data scientists and analysts.Responsibilities: Data Pipelines Data Integration️...

  • Data Engineer

    1 month ago


    London,, UK, United Kingdom Bolton Associates Full time

    We are working on behalf of an International (Re)Insurance client based in London to recruit a Lead/Senior Data Engineer, joining an established team of Analytics Engineers helping optimise the Actuarial teams' efficiency through data automation and the use of emerging technologies.The successful candidate will work on the following:- Actuarial Data...

  • Data Engineer

    4 weeks ago


    London,, UK, United Kingdom Vallum Associates Full time

    RequirementsAt least two years of professional experience as a data engineer with a proven ability to provide effective data solutions in a production setting.Knowledge of developing real-time data stream systems (ideally Kafka).Proven track record in developing data systems using PySpark and Apache Spark for batch processing.Capable of managing data intake...

  • Data Engineer

    4 weeks ago


    London,, UK, United Kingdom Vallum Associates Full time

    RequirementsAt least two years of professional experience as a data engineer with a proven ability to provide effective data solutions in a production setting.Knowledge of developing real-time data stream systems (ideally Kafka).Proven track record in developing data systems using PySpark and Apache Spark for batch processing.Capable of managing data intake...

  • Data Engineer

    1 month ago


    London,, UK, United Kingdom Hexegic Full time

    Data Engineers are responsible for aiding UK Health in developing a cleaned, validated, and discoverable data foundation to enable new analytical workflows and applications.They will need to work closely with UK Health subject matter experts to scope new data integrations, design a data model, and perform data cleansing and transformation within the data...

  • Data Engineer

    1 week ago


    London,, UK, United Kingdom Omnis Partners Full time

    Data Engineer Up to £70,000Healthcare Hybrid in Central London Join a global healthcare company in Central London as a Data Engineer, providing data solutions with a focus on ETL development, cloud deployment, and building data & analytics capabilities; offering up to £70,000, hybrid work, and a discretionary bonusCompanyA global Healthcare company host to...

  • Data Engineer

    4 weeks ago


    London,, UK, United Kingdom Understanding Recruitment Full time

    Fantastic new opportunity for a Data Engineer to join a high performing team at a leading hedge fund!You'll be vital in building and maintaining infrastructure for the efficient execution of their quantitative and discretionary trading strategies.What we'd love to see: MSc in Comp Sci, Engineering etc. from a Top University Demonstrable Data...

  • Data Engineer

    4 weeks ago


    London,, UK, United Kingdom Understanding Recruitment Full time

    Fantastic new opportunity for a Data Engineer to join a high performing team at a leading hedge fund!You'll be vital in building and maintaining infrastructure for the efficient execution of their quantitative and discretionary trading strategies.What we'd love to see: MSc in Comp Sci, Engineering etc. from a Top University Demonstrable Data...

  • Data Engineer

    1 month ago


    London,, UK, United Kingdom Seargin UK & Ireland Full time

    Data Engineers needed for 6-12 month project based in London paying £450 a day inside IR35. This is an Urgent role an ideally we need the data engineers to be able to start asap or within 3 weeks as a maximum. All data engineers have to able to work in London 3 days a week, have a valid visa as we Are not able to sponsorTechnical Stack:- Python, Postgres...