Data Engineering Expert for Large Scale Research Platform

2 weeks ago


London, Greater London, United Kingdom AlphaSights Full time

We are seeking a skilled Data Engineer to join our Engineering team in London, working on designing, developing, deploying, and supporting data infrastructure, pipelines, and architectures. If you're passionate about solving complex data challenges with code and collaborating with talented colleagues in a high-performance environment, this role is a great fit.

About the Role

This position involves contributing to an architectural vision that will scale up to be the world's leading research platform. You will be responsible for writing clean, efficient, and maintainable code that powers data pipelines, workflows, and data operations in a production environment.

  • Ship working code: Implement reliable, scalable, and well-tested solutions to automate data ingestion, transformation, and orchestration across systems using Python, SQL, and working with SQL/NoSQL databases.
  • Own data operations infrastructure: Manage and optimise key data infrastructure components within AWS, including Amazon Redshift, Apache Airflow for workflow orchestration and other analytical tools, ensuring performance, reliability, and scalability.
  • Maintenance and troubleshooting: Oversee configuration, monitoring, troubleshooting, and continuous improvement of our infrastructure to support delivering high-quality insights and analytics.
Requirements

The ideal candidate should have 5+ years of hands-on data engineering development experience with deep expertise in Python, SQL, and working with SQL/NoSQL databases. Additionally, they should be skilled in designing, building, and maintaining data pipelines, data warehouses, and leveraging AWS data services, ETL frameworks, and Big Data processing (Spark, Hive, Trino), and data streaming.

Salary and Benefits

The estimated salary for this position is £120,000 - £150,000 per annum, depending on experience. We offer a competitive benefits package, including health insurance, retirement plan, and generous paid time off.



  • London, Greater London, United Kingdom DeepMind Full time

    At DeepMind, we're pushing the boundaries of artificial intelligence to benefit society and humanity. We're seeking an exceptional Expert Researcher to join our team and contribute to groundbreaking fundamental research and deployment in large scale pre-training.About UsWe're a diverse team of scientists, engineers, and machine learning experts working...


  • London, Greater London, United Kingdom DALLAS VA RESEARCH CORPORATION Full time

    About the RoleWe are seeking a highly skilled Research Engineer to join our team at Dallas VA Research Corporation. As a Research Engineer, you will play a key role in designing and developing large-scale artificial intelligence (AI) systems from the ground up.Key ResponsibilitiesYou will work closely with our team to design, develop, and deploy large-scale...


  • London, Greater London, United Kingdom Oxford Knight Full time

    Data Engineer for Large-Scale Financial SystemsOxford Knight is one of the world's largest hedge funds, offering a challenging and rewarding opportunity for a data engineer to join their prestigious technology team. In this role, you will work closely with the Data Science team to deliver cutting-edge solutions to complex data challenges, driving business...


  • London, Greater London, United Kingdom Citigroup Inc. Full time

    Join Our Team:Are you passionate about developing innovative software solutions? Do you have experience with building and running large-scale platforms? We are looking for an experienced Software Engineering Lead to join our Generative AI team at Citigroup Inc. In this role, you will lead the design and development of high-quality, highly reliable software...


  • London, Greater London, United Kingdom ENGINEERINGUK Full time

    Job Summary: Large Scale Distributed Systems Expert Wanted!We're looking for an experienced Large Scale Distributed Systems Expert to join our team at ENGINEERINGUK. As a key member, you'll be responsible for designing and developing large-scale distributed systems on the AWS platform. If you have a strong background in software engineering best practices...


  • London, Greater London, United Kingdom Intelmatix Full time

    About UsWe are Intelmatix, a visionary AI company dedicated to unlocking business potential through intelligent solutions.Our MissionTo empower organizations by providing innovative AI-powered tools that drive growth and success.Job Title: Data Scientist / Machine Learning ScientistSalary Range: $100,000 - $160,000 per yearThis role involves working on...


  • London, Greater London, United Kingdom Spotify Full time

    Job DescriptionThis role is a unique opportunity to join the Speak team at Spotify, where we develop world-class speech technologies that power personalized generative voice products at scale.Responsibilities:Create large-scale speech and audio data pipelines using frameworks like Google Cloud Platform and Apache Beam.Develop machine learning projects that...


  • London, Greater London, United Kingdom Citigroup Global Markets Limited Full time

    Overview:Citigroup Global Markets Limited is seeking a highly skilled AI Engineer to join our team in London. As a key member of our Common Platform Engineering group, you will be responsible for designing and building high-quality, highly reliable software platforms using cutting-edge technologies such as LLMs.In this role, you will partner closely with...


  • London, Greater London, United Kingdom DeepMind Technologies Limited Full time

    Company OverviewDeepMind Technologies Limited is a pioneering leader in the field of artificial intelligence. Our team of scientists and engineers work together to advance the state of the art in AI, using our technologies for widespread public benefit and scientific discovery.Job SummaryWe are seeking an experienced Research Scientist to join our team and...


  • London, Greater London, United Kingdom Qube Research & Technologies Full time

    Qube Research & Technologies (QRT) is a cutting-edge quantitative and systematic investment manager operating globally in various liquid asset classes. As a technology-driven group, we implement a scientific approach to investing, combining data, research, and trading expertise to solve complex challenges. Our collaborative mindset enables us to innovate and...


  • London, Greater London, United Kingdom LexisNexis Intellectual Property Solutions Full time

    About the RoleWe are seeking a highly skilled GenAI Architect to join our AI Innovation Team at LexisNexis Intellectual Property Solutions. As a key member of our team, you will be responsible for designing, developing, and maintaining high-quality software solutions to support the Generative AI platform.Salary: £80,000 - £110,000 per annumCompany...


  • London, Greater London, United Kingdom Citigroup, Inc. Full time

    About the RoleWe are seeking a highly skilled Technical Lead to join our team in developing a green-field development at scale. As a key member of our engineering team, you will design and build high-quality, highly reliable software, partnering closely with other development teams, quants, and subject-matter experts.Key ResponsibilitiesDesign and build...

  • Senior Data Engineer

    2 weeks ago


    London, Greater London, United Kingdom Kainos Full time

    Kainos is a leading provider of digital services, and we're seeking a highly skilled Senior Data Engineer to join our team.About the RoleWe are looking for an experienced Senior Data Engineer to design and develop large-scale data processing software. As a key member of our agile team, you will work closely with architects and lead engineers to ensure that...


  • London, Greater London, United Kingdom DeepMind Technologies Limited Full time

    At DeepMind Technologies Limited, we value diversity of experience, knowledge, backgrounds and perspectives and harness these qualities to create extraordinary impact. We are committed to equal employment opportunity regardless of sex, race, religion or belief, ethnic or national origin, disability, age, citizenship, marital, domestic or civil partnership...


  • London, Greater London, United Kingdom LinuxRecruit Full time

    **About the Role:**We're looking for skilled Java engineers to join our team and work on a large-scale platform. This project involves building a low-latency platform on a Spring Boot framework and deploying it onto GCP infrastructure.**Responsibilities:**Develop a low-latency platform on a Spring Boot framework.Deploy onto GCP infrastructure.Work in a...

  • AI Research Engineer

    2 weeks ago


    London, Greater London, United Kingdom Microsoft Full time

    Microsoft AI is committed to training the world's most capable AI frontier models, pushing the boundaries of scale, performance, and product deployment. As a member of our Pre-Training team, you will be tackling some of the most challenging problems in deep learning at scale.About the RoleWe are looking for outstanding individuals who are excited about...


  • London, Greater London, United Kingdom Bloomberg Full time

    Bloomberg Data Engineer Job DescriptionWe are responsible for delivering data, news, and analytics through innovative technology, quickly and accurately, by combining data and context to paint a complete picture for our clients. Our team collects, analyzes, processes, and publishes the data that forms the backbone of our Bloomberg Terminal.As a Data...


  • London, Greater London, United Kingdom Connexity Full time

    We are a cutting-edge digital marketing company, Skimlinks, driving e-commerce success for top online retailers.Our innovative approach combines big data with large-scale machine learning to create a fundamentally new approach to digital marketing.We collect data on over 1 billion users per month and analyze hundreds of millions of documents daily.As a Data...


  • London, Greater London, United Kingdom Spotify Full time

    About the RoleAs a key member of our team, you will design and build large-scale data pipelines using data processing frameworks like Scio, BigQuery, Google Cloud Platform, and Apache Beam.You'll develop, deploy, and operate Java services that significantly impact millions of users. Your expertise will help power machine learning projects, making...


  • London, Greater London, United Kingdom University of Bristol Full time

    As a Software Security Specialist at the University of Bristol, you will be part of our SCULI research programme, focusing on securing ultra-large scale infrastructures. Your primary responsibility will be conducting experimental work on software security, including instrumenting systems, developing proof-of-concept prototypes, and testing them in...