Senior Engineer, Data Engineering
1 week ago
Contract : £500 Per Day
Our client is seeking a Data Engineer with a solid foundation in both data engineering and business intelligence, capable of understanding and addressing business needs. This role involves managing and optimizing AWS-based data processes, orchestrating data flows, and supporting business intelligence capabilities. Data Pipeline Management :
Database Integration : Maintain MySQL databases on AWS (approx. 25TB) as primary data sources feeding the data pipeline.
Real-Time Data Ingestion : Utilize Apache Kafka to manage data flows from MySQL into AWS S3 buckets, enabling real-time streaming.
Data Processing : Implement ETL processes using AWS Glue and PySpark to build data lakes within S3 for further analysis.
Business Intelligence (BI) :
Facilitate data availability for BI purposes, supporting daily updates and aiming to move from overnight batch jobs to near-real-time 15-minute refreshes.
Collaborate on BI dashboard and reporting creation to provide actionable insights.
Complex Data Reconciliation :
Reconcile transaction records across multiple merchants, currencies, and network operators, addressing both straightforward and complex data relationships.
Data Engineering Expertise :
Proficiency with MySQL , Apache Kafka , AWS Glue , PySpark , and AWS S3 for managing data pipelines and ETL processes.
Business Intelligence Skills :
Experienced in designing and maintaining BI reports and dashboards with frequent data refresh capabilities.
Data Reconciliation Experience :
Skilled in reconciling transactional data across diverse systems, operators, and currencies to ensure data accuracy.
Ability to translate business requirements into data solutions and work collaboratively with non-technical stakeholders to develop reports and dashboards tailored to business needs.
Strong communication skills to bridge technical and business needs, working closely with stakeholders to ensure data solutions support strategic objectives.
-
Senior Engineer, Data Engineering
2 weeks ago
United Kingdom Understanding Recruitment Full timeSenior Data Engineer | 1x per month in the London office (flex) I am looking for a Senior Data Engineer to join a cutting edge start-up that's leading the charge in IoT, and data-driven tech across Europe! They are revolutionising the automotive space with cutting-edge in-car technology, web apps, and APIs. To accelerate the building of their...
-
Senior Engineer, Data Engineering
1 month ago
United Kingdom Revolution Technology Full timeSenior Data Engineer (Azure) | London or Coventry Hybrid | up to £85,000 Product led B2B SaaS company focussed on making the property and construction space data driven, price efficient and more sustainable. Joining as the first Senior Data Engineer Senior you will have full ownership over the design and build of their modern, scalable, and automated...
-
Senior Engineer, Data Engineering
1 month ago
United Kingdom Revolution Technology Full timeSenior Data Engineer (Azure) | London or Coventry Hybrid | up to £85,000 Product led B2B SaaS company focussed on making the property and construction space data driven, price efficient and more sustainable. Joining as the first Senior Data Engineer Senior you will have full ownership over the design and build of their modern, scalable, and automated...
-
Senior Engineer, Data Engineering
1 month ago
United Kingdom Revolution Technology Full timeSenior Data Engineer (Azure) | London or Coventry Hybrid | up to £85,000 Product led B2B SaaS company focussed on making the property and construction space data driven, price efficient and more sustainable. Joining as the first Senior Data Engineer Senior you will have full ownership over the design and build of their modern, scalable, and automated...
-
Senior Data Engineer
3 weeks ago
united kingdom Xcede Group Full timeSenior Data Engineer Up to £92,500 + Bonus Fully Remote in UK OVERVIEW Major FTSE100 retail giant hiring for Senior Data Engineers, to lead small teams of Data Engineers on key projects, directly line managing them and providing technical leadership and guidance to the team. You will be joining a well-established Data function with huge customer...
-
Senior Engineer, Data Engineering
2 weeks ago
United Kingdom Harnham Full time €80,000Senior Data Engineer (AWS, Snowflake, Spark) Licensing & IP Rights - Mayfair, London (Hybrid, 1-2 days per week) We're partnered with a SaaS company pioneering data solutions within the IP licensing industry. For the past 6-7 years, they have been tackling the challenge of data transparency in licensing, providing companies with essential insights to...
-
Senior Engineer, Data Engineering
1 week ago
United Kingdom F5 Consultants Full timeJ ob Title: Senior Data Engineer - Palantir Location: Hybrid - home working with travel to client site when necessary They are seeking Senior Data Engineers with 3+ years experience of working with Palantir. Experience in developing data pipelines, adhering to Data Engineering best practices (coding, unit testing, version control, code...
-
Senior Engineer, Data Engineering
4 weeks ago
United Kingdom Prism Digital Full time €550Senior Azure Data Engineer | PySpark (Python), Synapse, Data Lake | Tech for Good | Fully Remote Location: Remote Azure – Solid experience required of the Azure Data ecosystem Python - ESSENTIAL as PySpark is used heavily. Azure Data Lake / Data Bricks / Data Factory Be happy to act as a lead and mentor to the other permanent Azure Data...
-
Senior Engineer, Data Engineering
1 month ago
United Kingdom Prism Digital Full timeSenior Azure Data Engineer | PySpark (Python), Synapse, Data Lake | Tech for Good | Fully Remote Location: Remote Azure – Solid experience required of the Azure Data ecosystem Python - ESSENTIAL as PySpark is used heavily. Azure Data Lake / Data Bricks / Data Factory Be happy to act as a lead and mentor to the other permanent Azure Data...
-
Senior Engineer, Data Engineering
1 month ago
United Kingdom Prism Digital Full time €550Senior Azure Data Engineer | PySpark (Python), Synapse, Data Lake | Tech for Good | Fully Remote Location: Remote Azure – Solid experience required of the Azure Data ecosystem Python - ESSENTIAL as PySpark is used heavily. Azure Data Lake / Data Bricks / Data Factory Be happy to act as a lead and mentor to the other permanent Azure Data...
-
Senior Engineer, Data Engineering
1 week ago
United Kingdom Vecta Search Full timeA well-established regional Insurance Broker based in Surrey is looking for a Senior Data Engineer to join their growing team. This is a fantastic opportunity to develop a brand-new Enterprise Data Warehouse from scratch, using cutting-edge technologies. As the Senior Data Engineer, you will play a key role in designing, implementing, and maintaining data...
-
Senior Engineer, Data Engineering
1 week ago
United Kingdom Vecta Search Full timeA well-established regional Insurance Broker based in Surrey is looking for a Senior Data Engineer to join their growing team. This is a fantastic opportunity to develop a brand-new Enterprise Data Warehouse from scratch, using cutting-edge technologies. As the Senior Data Engineer, you will play a key role in designing, implementing, and maintaining data...
-
Senior Engineer, Data Engineering
4 weeks ago
United Kingdom Prism Digital Full timeSenior Azure Data Engineer | PySpark (Python) & Synapse | Tech for Good | Fully Remote Azure – Solid experience required of the Azure Data ecosystem Python - ESSENTIAL as PySpark is used heavily. Azure Data Lake / Data Bricks / Data Factory Be happy to act as a lead and mentor to the other permanent Azure Data Engineers Our client is a...
-
Senior Engineer, Data Engineering
1 month ago
United Kingdom Prism Digital Full timeSenior Azure Data Engineer | PySpark (Python) & Synapse | Tech for Good | Fully Remote Azure – Solid experience required of the Azure Data ecosystem Python - ESSENTIAL as PySpark is used heavily. Azure Data Lake / Data Bricks / Data Factory Be happy to act as a lead and mentor to the other permanent Azure Data Engineers Our client is a...
-
Senior Engineer, Data Engineering
1 month ago
United Kingdom Prism Digital Full timeSenior Azure Data Engineer | PySpark (Python) & Synapse | Tech for Good | Fully Remote Azure – Solid experience required of the Azure Data ecosystem Python - ESSENTIAL as PySpark is used heavily. Azure Data Lake / Data Bricks / Data Factory Be happy to act as a lead and mentor to the other permanent Azure Data Engineers Our client is a...
-
Senior Engineer, Data Engineering
3 weeks ago
United Kingdom Synchro Full timeHave You Thought Of Relocating To Dubai? Senior Data Engineer – Relocate to Dubai! – Full Time – Hybrid Working Are you a skilled Senior Data Engineer with expertise in Python, SQL, and PySpark ? Are you excited to work in a fast-paced financial sector with opportunities for growth and cutting-edge projects? This is your chance to relocate to...
-
Senior Engineer, Data Engineering
4 weeks ago
United Kingdom Synchro Full timeHave You Thought Of Relocating To Dubai? Senior Data Engineer – Relocate to Dubai! – Full Time – Hybrid Working Are you a skilled Senior Data Engineer with expertise in Python, SQL, and PySpark ? Are you excited to work in a fast-paced financial sector with opportunities for growth and cutting-edge projects? This is your chance to relocate to...
-
Senior Engineer, Data Engineering
1 month ago
United Kingdom Harnham Full timeSenior Data Engineer Location: Remote, Oxford (Once per Month) Salary: £82k + Benefits Overview: We are looking for a Senior Data Engineer to play a key role in evolving our data infrastructure, ensuring enhanced reliability and scalability. Reporting to the Data & Analytics Director, you will be integral in driving forward our data...
-
Senior Engineer, Data Engineering
1 month ago
United Kingdom Harnham Full timeSenior Data Engineer Location: Remote, Oxford (Once per Month) Salary: £82k + Benefits Overview: We are looking for a Senior Data Engineer to play a key role in evolving our data infrastructure, ensuring enhanced reliability and scalability. Reporting to the Data & Analytics Director, you will be integral in driving forward our data...
-
Senior Engineer, Data Engineering
1 month ago
United Kingdom Harnham Full timeSenior Data Engineer Location: Remote, Oxford (Once per Month) Salary: £82k + Benefits Overview: We are looking for a Senior Data Engineer to play a key role in evolving our data infrastructure, ensuring enhanced reliability and scalability. Reporting to the Data & Analytics Director, you will be integral in driving forward our data...