Senior Data Engineer
1 week ago
Job Title: Data Engineer
Location: Remote
Contract: £500 Per Day
Role Overview
Our client is seeking a Data Engineer with a solid foundation in both data engineering and business intelligence, capable of understanding and addressing business needs. This role involves managing and optimizing AWS-based data processes, orchestrating data flows, and supporting business intelligence capabilities. The ideal candidate will not only bring technical expertise but also a strong business acumen to facilitate effective collaboration with non-technical stakeholders.
Key Responsibilities
- Data Pipeline Management:
- Database Integration: Maintain MySQL databases on AWS (approx. 25TB) as primary data sources feeding the data pipeline.
- Real-Time Data Ingestion: Utilize Apache Kafka to manage data flows from MySQL into AWS S3 buckets, enabling real-time streaming.
- Data Processing: Implement ETL processes using AWS Glue and PySpark to build data lakes within S3 for further analysis.
- Business Intelligence (BI):
- Facilitate data availability for BI purposes, supporting daily updates and aiming to move from overnight batch jobs to near-real-time 15-minute refreshes.
- Collaborate on BI dashboard and reporting creation to provide actionable insights.
- Complex Data Reconciliation:
- Reconcile transaction records across multiple merchants, currencies, and network operators, addressing both straightforward and complex data relationships.
Technical Requirements
- Data Engineering Expertise:
- Proficiency with MySQL, Apache Kafka, AWS Glue, PySpark, and AWS S3 for managing data pipelines and ETL processes.
- Business Intelligence Skills:
- Experienced in designing and maintaining BI reports and dashboards with frequent data refresh capabilities.
- Data Reconciliation Experience:
- Skilled in reconciling transactional data across diverse systems, operators, and currencies to ensure data accuracy.
Business & Communication Skills
- Business Acumen:
- Ability to translate business requirements into data solutions and work collaboratively with non-technical stakeholders to develop reports and dashboards tailored to business needs.
- Communication:
- Strong communication skills to bridge technical and business needs, working closely with stakeholders to ensure data solutions support strategic objectives.
-
Senior Data Engineer
1 month ago
UK, UK, United Kingdom Understanding Recruitment Full timeSenior Data Engineer | 1x per month in the London office (flex)I am looking for a Senior Data Engineer to join a cutting edge start-up that’s leading the charge in IoT, and data-driven tech across Europe!They are revolutionising the automotive space with cutting-edge in-car technology, web apps, and APIs. To accelerate the building of their greenfield...
-
Senior Data Engineer
4 weeks ago
UK, UK, United Kingdom Understanding Recruitment Full timeSenior Data Engineer | 1x per month/quarter in the London office (flex)I am looking for a Senior Data Engineer to join a cutting edge start-up that’s leading the charge in IoT, and data-driven tech across Europe!They are revolutionising the automotive space with cutting-edge in-car technology, web apps, and APIs. To accelerate the building of their...
-
Senior Data Scientist
1 week ago
London, UK, UK, United Kingdom Data Idols Full timeSenior Data Scientist - LoyaltySalary: £80k - £90k Location: London, HybridData Idols are working with one of the top global retail companies to expand its data capabilities by hiring a Senior Data Scientist within its loyalty scheme department. The Opportunity You will be working on various projects across loyalty, including customer segmentation,...
-
Senior Data Engineer
1 week ago
UK, UK, United Kingdom Rekall Consulting Full timeJob Title: Senior Backend/Data EngineerDescription:We are an innovative, Series A startup in the data space seeking an expert Senior Backend/Data Engineer to join our core team. This role is focused on transforming raw data into valuable asset inventory insights—a key responsibility typically handled by data engineers. You’ll work with cutting-edge...
-
Senior Python Data Engineer
1 week ago
UK, UK, United Kingdom Oliver Bernard Full timeSenior Python Data Engineer - Python, AWS, ETLMy client who are in the Slot and Casino Games industry, have a brand new headcount for 4 Python/Data Engineers to come and join their rapidly growing engineering team on a Fully Remote basis!Key Skills and Experience:Prior experience in a Python Data Engineering roleStrong coding skills in PythonSQL...
-
Senior Data Engineer
1 month ago
UK, UK, United Kingdom Harnham Full timeSenior Data EngineerLocation: Remote working (Once a month in Oxford)Salary: £80k + competitive benefits packageRole Overview:Our client is seeking a Senior Data Engineer to elevate their data infrastructure's reliability and scalability. Reporting to the Data & Analytics Director, you will drive the development of a larger data mesh setup while...
-
Senior Data Engineer
1 month ago
UK, UK, United Kingdom Harnham Full timeSenior Data Engineer Location: London (3 Days a week in office) Salary: £90,000Company:Our client are in the financial space and currently undertaking a greenfield build of a Snowflake data warehouse on Azure, aimed at creating a more streamlined, user-friendly data environment. This role will work closely with a data architect, senior FP&A manager to...
-
Senior Data Engineer
1 week ago
London, UK, UK, United Kingdom Harnham Full timeSenior Data EngineerLondon2 days a week in OfficeTHE COMPANYThis company was founded over 20 years ago and is a global leading technology and private equity firm. The company is made up of roughly 50 employees and they are looking now looking to expand.THIS ROLEAs a Senior Data Engineer, you will be…Designing and developing scalable and reliable pipelines...
-
Senior Data Engineer
2 weeks ago
UK, UK, United Kingdom Kainos Full timeWhen you join Kainos, you get to think beyond limitations to make an immediate and positive impact – like transforming digital services for millions of citizens or helping Fortune 500 companies get live and thrive on Workday.You’ll be part of a people-first culture that is growing around the world. We’re a creative, committed, and diverse group of...
-
Senior Data Platform Engineer
2 weeks ago
UK, UK, United Kingdom Amicus Full timeSenior Data Platform Engineer – Python / AWS / Data Platforms – Remote I am partnered with an European BioTech start-up having secured their Series A round of funding they are looking to expand their Engineering steam with addition of multiple mid-Senior engineers.CompanyThe business specialises in building tools and systems to further the drug discovery...
-
Senior Data Engineer
1 week ago
UK, UK, United Kingdom MRJ Recruitment Full timeRevolutionise healthcare with a leading HealthTech SaaS brand that is revolutionising the way GPs operate.As a Senior Data Engineer, you'll design, develop, and maintain robust data pipelines on AWS. Optimise data warehouse performance, ensure data integrity, and enable large-scale analytics. Contribute to a platform that streamlines workflows, enhances...
-
Senior Data Engineer
4 months ago
UK, UK, United Kingdom TLP Consultancy Ltd Full time£65,000 to £75,000 Permanent Near Epsom, Surrey (location is most easily reached by own transport) Hybrid working (3 days a week in the office) Generous benefits package (Performance related bonus, Pension, access to their car scheme, Birthday voucher, Workplace Nursery Benefit & Cycle Scheme, holiday purchase scheme, discounted onsite gym membership,...
-
Senior Data Engineer
1 week ago
UK, UK, United Kingdom WeDo Full timeWeDo has partnered with a leading fintech scale-up that is looking to scale it's Data Engineering Practice.Who are they.......Imagine a finance app that’s like your smart, low-fee wingman—budgeting help, savings boosts, and no confusing lingo! It’s here to make managing money actually easyWith it's significant growth in the past few years and...
-
Senior Data Engineer
1 week ago
UK, UK, United Kingdom Digital Gurus Full timeDigital Gurus has partnered with a Tech for Good business, on an exciting Azure Data Journey. The business has undergone huge amounts of transformation and change. With Data at the heart of what they do. For this next phase of the journey, they are looking for an Azure Data Engineer to join them!As they look to invest and integrate further, this is a great...
-
Senior Data Scientist
2 weeks ago
UK, UK, United Kingdom Harnham Full timeSENIOR DATA SCIENTISTUP TO £80,000REMOTE (UK)COMPANY:We are partnered with a global player in data-driven solutions for a high-demand sector, partnering with over 150 organizations worldwide - mainly within telecoms. The company specialises in optimising revenue streams through innovative solutions, including data clearing, strategic insights, and IoT...
-
Senior Data Analyst
1 month ago
UK, UK, United Kingdom developrec Full timeSenior Data Analyst, Central London (2 days per week), £70k - £80k + Bonus & Benefits!An exciting opportunity to join my client’s team and make an impact on the future of data in finance. As a Senior Data Analyst, you will be instrumental in advancing AI/ML solutions, data governance practices, and financial data analysis. Collaborating with Data,...
-
Senior Data Architect
1 month ago
UK, UK, United Kingdom Marx Enterprise Transformation Architecture Full timeJob Title: Senior Data ArchitectType: Remote, Full-timeAbout the RoleMarx Technology Consulting Ltd is looking for a Senior Data Architect to join our dynamic and innovative team. Based in London, we are a leading enterprise architecture consulting firm committed to helping clients harness the power of data for informed decision-making and strategic growth....
-
Senior Data Engineer Python Spark AWS
1 month ago
London, UK, UK, United Kingdom Client Server Full timeSenior Data Engineer (Python Spark AWS) Remote UK to £90kAre you a tech savvy Data Engineer with strong Python coding skills?You could be progressing your career in a senior, hands-on Data Engineer role as part of a friendly and supportive international team at a growing and hugely successful European car insurance tech company as they expand their UK...
-
Senior Software Engineer
4 days ago
UK, UK, United Kingdom Understanding Recruitment Full timeSenior Software Engineer (Data Engineering)Join a dynamic team shaping the future of financial markets with cutting-edge trade surveillance technology. I am seeking a Senior Software Engineer with data engineering experience to build scalable, robust data pipelines in cloud environments and tackle challenges like data velocity, schema management, and high...
-
Senior Data Engineer
1 week ago
London, UK, UK, United Kingdom MRJ Recruitment Full timeMRJ Recruitment is partnering with a high-growth startup revolutionising the service industry with an AI-powered platform that’s reshaping appointment management and customer relationships. Having already turned heads in the tech world, they’re now entering an exciting phase of growth—and they need a talented Senior Data Engineer like you to make it...