Data Engineering Platform Lead
3 weeks ago
The role of the Senior Data Engineer will be to design and implement a cutting-edge internal data ingestion platform, leveraging Apache Spark, Kafka, and Airflow. This platform will enable scalable and efficient ingestion of diverse datasets, supporting downstream analytics and visualisation requirements.
The successful candidate will lead a team of data engineers, driving the development of configurable Spark ETL pipelines and leveraging modern data architecture principles. They will act as the product owner for the internal data ingestion platform, ensuring alignment with business needs and long-term scalability.
This is a high-impact role that requires expertise in designing scalable systems to handle diverse datasets efficiently. The ideal candidate will have extensive experience with Apache Spark, Kafka, and Airflow, as well as proficiency in designing and deploying solutions on Azure Kubernetes Service (AKS).
**Key Responsibilities:**
- Design and implement a robust data ingestion platform using Apache Spark, Kafka, and Airflow.
- Lead a team of data engineers to develop configurable Spark ETL pipelines and ensure adherence to best practices.
- Act as the product owner for the internal data ingestion platform, ensuring alignment with business needs and long-term scalability.
- Collaborate with cross-functional teams to ensure seamless integration with downstream analytics and visualisation tools.
**Required Skills and Qualifications:**
- Extensive experience with Apache Spark, Kafka, and Airflow for data ingestion and orchestration.
- Proficiency in designing and deploying solutions on Azure Kubernetes Service (AKS).
- Deep understanding of Databricks for data preprocessing and transformation.
- Hands-on experience with Snowflake, SQL warehouses, and Delta Lake for data storage.
**Estimated Salary: $150,000 - $200,000 per annum.
-
Data Platform Architect
4 weeks ago
London, Greater London, United Kingdom Data Idols Full time £80,000 - £90,000Company OverviewData Idols is a prestigious retailer that's evolving into a digital-first, data-led organisation. We're seeking a talented Senior Data Platform Engineer to join our team and contribute directly to data-driven decision-making, helping millions of customers and thousands of colleagues alike.Job DescriptionWe're looking for a skilled Senior Data...
-
Data Platform Architect
3 weeks ago
London, Greater London, United Kingdom Data Idols Full time £80,000 - £90,000Data Idols is a prestigious retailer evolving into a digital-first, data-led organisation. We are seeking a Senior Data Platform Engineer to shape the data landscape and work alongside a talented team to build, optimise, and secure scalable data infrastructure.As a Senior Data Platform Engineer at Data Idols, you will play a vital role in designing,...
-
Data Platform Architect
3 weeks ago
London, Greater London, United Kingdom Data Idols Full time £80,000 - £90,000About Data IdolsWe are a prestigious retailer that's evolving into a digital-first, data-led organisation. As a Senior Data Platform Engineer, you'll be pivotal to shaping the data landscape and work alongside a talented, collaborative team to build, optimise, and secure scalable data infrastructure.Reporting to the Head of Data Platforms, you'll have the...
-
Senior Data Platform Engineer
3 weeks ago
London, Greater London, United Kingdom CAPGEMINI ENGINEERING Full timeAbout the JobThis is an exceptional opportunity to join CAPGEMINI ENGINEERING as a Senior Data Platform Engineer, working on innovative data architecture projects that drive business growth. With a strong background in data engineering, AI, and platform consulting, you will deliver cutting-edge data architecture solutions to clients, leveraging your...
-
Data Platform Architect
3 weeks ago
London, Greater London, United Kingdom Data Idols Full time £80,000 - £90,000Job OverviewData Idols is seeking a highly skilled Senior Data Platform Engineer to join our team. This role will play a pivotal part in shaping the data landscape and working alongside a talented team to build, optimise, and secure scalable data infrastructure.
-
Data Engineering Platform Lead
7 days ago
London, Greater London, United Kingdom NewDay Ltd Full timeWe are seeking an experienced Data Engineering Platform Lead to join our team at NewDay Ltd in London. This is a permanent full-time role that offers a competitive salary, bonus opportunities, and a hybrid working approach. In this position, you will lead a team of platform engineers to design, optimize, and maintain our core technology platforms, ensuring...
-
Data Platform Specialist
7 days ago
London, Greater London, United Kingdom Monte Carlo Data, Inc. Full time**About Monte Carlo Data, Inc.**As businesses increasingly rely on data to drive informed decisions, it's crucial that this data is accurate and reliable. Monte Carlo, the industry-leading data reliability company, has created the first end-to-end Data Observability platform. We've achieved numerous accolades, including Best Workplace for 2024 and a DBTA...
-
Data Platform Strategist
4 weeks ago
London, Greater London, United Kingdom Data Science Festival Full timeJob Description:We are seeking a highly skilled Data Platform Strategist to lead the development and delivery of our data platform. As a key member of the Data Tech team, you will drive responsible usage of our newly built data platform, ensuring it evolves with cutting-edge technologies and meets the needs of practitioners and business partners.The...
-
Cloud Data Engineering Lead
4 weeks ago
London, Greater London, United Kingdom Data Idols Full time £80,000 - £90,000About the RoleWe are currently seeking a Senior Data Platform Engineer to join our team at Data Idols, a renowned retailer in the digital-first era. This is an exciting opportunity to contribute to the design, implementation, and maintenance of a robust data platform essential to our ongoing digital transformation.Your Key Responsibilities:Design and...
-
Cloud Data Engineering Lead
2 weeks ago
London, Greater London, United Kingdom Data Science Festival Full timeAbout the OpportunityAt the Data Science Festival, we are committed to driving innovation and excellence in data engineering. We are seeking a Cloud Data Engineering Lead to join our team and play a key role in shaping our data strategy.Key ResponsibilitiesDevelop and implement data pipeline architectures using modern cloud technologiesLead cross-functional...
-
Data Platform Lead
2 weeks ago
London, Greater London, United Kingdom ZipRecruiter Full timeJob Summary:A challenging role for a skilled data engineer to lead the development of our cloud-based analytics platform.The ideal candidate will have experience in building scalable data platforms using Python, SQL, and GCP.We are seeking a highly pragmatic engineer to work closely with our analytics team to build platforms that provide business critical...
-
Data Engineer
2 weeks ago
London, Greater London, United Kingdom Data Science Festival Full timeWe are currently seeking a Senior Data Platform Engineer to join our prestigious retailer that's evolving into a digital-first, data-led organisation. In this role, you'll be pivotal to shaping the data landscape and will work alongside a talented, collaborative team to build, optimise, and secure scalable data infrastructure.Estimated Salary: 80,000 –...
-
Data Platform Lead
6 days ago
London, Greater London, United Kingdom Canada Life Limited Full timeAre you an expert in data architecture looking for a new challenge? We're seeking a Data Platform Lead to join our Digital and Customer Engineering department at Canada Life UK. In this role, you'll be responsible for designing and implementing our data platform solution architecture using full-stack Azure data platform capabilities.About the RoleThis is an...
-
Data Engineering Platform Lead
4 weeks ago
London, Greater London, United Kingdom Core Consultants Full timeJob Title: Data Engineering Platform LeadAbout Us:
-
Data Platform Lead
2 weeks ago
London, Greater London, United Kingdom Oliver Bernard Full time £100,000 - £110,000At Oliver Bernard, we are seeking a talented Data Platform Lead to join our market analytics platform team.Key Responsibilities:Lead the team building our data platformWork closely with the analytics team to build platforms for business-critical insightsRequirements:Proven experience in leading data engineering teamsStrong skills in Python, SQL, and...
-
Cloud Data Engineer
2 weeks ago
London, Greater London, United Kingdom Data Science Festival Full timeAbout Our Data Science FestivalJob Title: Senior Data Platform EngineerWe are a leading digital-first, data-led organisation. As a Senior Data Platform Engineer, you'll be pivotal to shaping the data landscape and work alongside a talented, collaborative team to build, optimise, and secure scalable data infrastructure.In this role, you'll have the...
-
Cloud Data Platform Lead
7 days ago
London, Greater London, United Kingdom Ogury Ltd Full timeOgury Ltd, a global adtech company, is seeking an experienced Cloud Data Platform Lead to shape the future of our data platform.About the RoleThis is an exciting opportunity to define the long-term vision and lead execution of Ogury's data platform, making data available to all teams and enabling them to take well-informed data-driven...
-
Data Infrastructure Lead
5 days ago
London, Greater London, United Kingdom Data Idols Full timeData Idols is driving its digital transformation, and we need a talented Senior Data Platform Engineer to shape our data infrastructure. You'll design and implement scalable, reliable, and secure data platforms using Azure Services.We're looking for someone with a strong understanding of DevOps practices, solid knowledge of security best practices on Azure,...
-
Proprietary Data Platform Lead
7 days ago
London, Greater London, United Kingdom LinuxRecruit Full timeProprietary Data Platform LeadLinuxRecruit is seeking an experienced Proprietary Data Platform Lead to join our team in London.The successful candidate will be responsible for leading the systems engineering team, focusing on building a proprietary data platform and supporting fibre product development.The ideal candidate will have hands-on experience in...
-
Cloud Data Engineer Leader
3 weeks ago
London, Greater London, United Kingdom Data Idols Full time £80,000 - £90,000Job DescriptionWe are currently seeking a Senior Data Platform Engineer to join our team. This role involves designing and implementing scalable, reliable, and secure data platform infrastructure using Azure Services.Maintaining an optimised platform by monitoring performance across compute and storage solutionsLeading CI/CD deployments for data...