Data Engineering Team Lead

Found in: Talent UK C2 - 2 weeks ago

London, United Kingdom Trainline Full time

Job Description

As a Data Engineering Team Lead at Trainline you will

A committed lead and coach to an agile team of polyglot Data Platform Engineers building a cuttingedge data lakehouse platform on AWS using Kafka and cloudnative streaming technologies Iceberg Trinio DataHub and AirflowBe a brilliant people manager for the Data Engineering function that motivates and engages their team to develop their skills and increase their impactDeliver the platform to enable deliver world class data products including the latest in LLM and other AI technologiesLead the technical direction of the crossfunctional team making good choices on technologies and approach to get the biggest impact for the least riskFoster an obsession with quality and engineering excellence through automated repeatable processes using CICD TDD BDDDrive incremental growth in dev ops maturity building tools and practices that allow repeatable and efficient delivery of models to production with strong governance in the workflow and continuous monitoring of models in productionOwn the operation of the products built by your team and continuously improve operation performanceLead and coach a selforganised Agile team and continuously improve agile maturity and delivery predictabilityQualifications

Wed love to hear from you if you

Thrive in a diverse open and collaborative environmentExpert in JVM technologies primarily Scala but ideally with a working knowledge of Clojure or Java and a familiarity with other key languages especially PythonExpert in key data engineering platforms such as Kafka or other streaming technologies data lakes AWS S3 Iceberg Parquet analytics technologies Trinio Spark automation technologies Airflow ML Flow and data governance DataHubPeople management and technical leadership experienceAre passionate about agile software delivery with a track record of leading effective agile and lean software teamsA consistent background in software development in high volume environmentsHave a strong background in Dev Ops deploying managing and maintaining services using Docker Terraform and AWS CLI tools to achieve infrastructureascode and automated deploymentsHave an excellent working knowledge of AWS services EMR ECS IAM EC2 S3 DynamoDB MSK

Our Technology Stack

Scala and PythonKafka Spark Kafka Streams Kinesis Akka and KSQLAWS S3 Iceberg Parquet Glue and SparkEMR for our Data LakeElasticsearch Dynamodb and RedisStarburst and AthenaAirflow and ML Flow

  • Lead Data Engineer

    2 weeks ago

    City of London, United Kingdom Data Idols Full time

    **Lead Data Engineer** Data Idols are proud to be working with an iconic UK brand to assist them in hiring a Lead Data Engineer as they undertake a digital and data-driven journey that will transform the business. Data is at the heart of the organisation and forms a key part oftheir future plans. **Skills and Experience**: - Awareness and understanding of...

  • Lead Engineer/Data Engineer

    3 weeks ago

    London, United Kingdom Abs Data Full time

    Lead Data Engineer Do you enjoy creating valuable insights with data? Do you want to play a key role in transforming our firm into an agile organization? Being agile will make us more responsive, more adaptable, and ultimately more innovative. We’re looking for a Data Engineer to: ~ transform data into valuable insights that inform business...

  • Lead Data Engineer

    23 hours ago

    London, United Kingdom Harnham - Data & Analytics Recruitment Full time

    Lead Data Engineer - Residential Property Company Location: London (flexible working available) **Salary**: Competitive We are a leading residential property company looking to hire a Lead Data Engineer to join our growing data-focused team. You will be responsible for leading a team of data engineers and playing a key role in the development and...

  • Lead Data Engineer

    3 weeks ago

    London, United Kingdom Harnham - Data & Analytics Recruitment Full time

    **LEAD DATA ENGINEER £70,000 - £75,000 + BENEFITS HYBRID - LONDON OR MAIDENHEAD** Are you a Senior Data Engineer looking for an exciting new role working within a team of likeminded individuals? Do you want to become an integral part of a team working within a betting company? If this is the role you have been searching for read on! **THE...

  • Data Engineering Manager

    2 days ago

    London, United Kingdom Data Idols Full time

    **Data Engineering Manager** Data Idols are proud to be working with an iconic UK brand to assist them in hiring a Data Engineering Manager as they undertake a digital and data-driven journey that will transform the business. Data is at the heart of the organisation and forms a keypart of their future plans. **The Opportunity** As a Data Engineering...

  • Data Engineer

    5 days ago

    London, United Kingdom Data Idols Full time

    **Data Engineer** Data Idols are proud to be working with a leading technology and engineering consultancy who have been running for decades globally. They work with a variety of clients across different industries on their biggest challenges. Would you like to work alongsidethis company, where you get the chance to work across a wide range of tools, tech,...

  • Lead Data Engineer

    Found in: Talent UK 2A C2 - 2 weeks ago

    London, United Kingdom ON Data Staffing Full time

    We are seeking a dynamic and experienced Databricks professional to join our client, a growing startup consultancy, as a Lead Data Engineer.As Lead Data Engineer, you will play a pivotal role in driving the success of our client's Databricks practice through your expertise in presales, strong technical acumen, consulting skills, and effective team...

  • Data Engineering

    3 weeks ago

    London, United Kingdom Areti Group Full time

    Data Team Lead - GIS/Geospatial Data Work Model: Hybrid, 2 or 3 days a week Areti Group have partnered with a powerhouse in the Data and Product space that specializes in Location Intelligence and Spatial Data, who are seeking a Data Delivery Lead/Head of Data Delivery to join a newly formed team within their business. 15 people) responsible for the...

  • Data Engineer

    6 days ago

    London, United Kingdom Data Idols Full time

    **Data Engineer** Data Idols are excited to be working with one of the top 100 companies in the world to recruit a Data Engineer! A company which owns multiple brands and who are investing heavily into 'all things data' to revolutionise their extraordinary team. **Skills and Experience**: - Experience with Azure - Experience with SQL - Experience with...

  • Senior Data Engineer

    5 days ago

    London, United Kingdom Data Idols Full time

    **Senior Data Engineer** Data Idols are proud to be working with a leading technology and engineering consultancy who have been running for decades globally. They work with a variety of clients across different industries on their biggest challenges. Would you like to work alongsidethis company, where you get the chance to work across a wide range of tools,...

  • Senior Engineer, Data Engineering

    Found in: Jooble UK C2 - 3 days ago

    London, United Kingdom Data Idols Full time

    Senior Data Engineer Location: London (Hybrid) Data Idols are working with one of the fast-growing companies in the logistics sector that are making the world a greener place. They are looking for a Senior Data Engineer to join their team within the Data Space. As a Senior Data Engineer, you will work alongside different data teams to design,...

  • Senior Engineer, Data Engineering

    3 days ago

    London, United Kingdom Data Idols Full time

    Senior Data Engineer Location: London (Hybrid) Data Idols are working with one of the fast-growing companies in the logistics sector that are making the world a greener place. They are looking for a Senior Data Engineer to join their team within the Data Space. As a Senior Data Engineer, you will work alongside different data teams to design,...

  • Data Engineer

    2 weeks ago

    City of London, United Kingdom Data Idols Full time

    **Senior Data Engineer** Data Idols are proud to be working with an iconic UK brand to assist them in hiring a Senior Data Engineer as they undertake a digital and data-driven journey that will transform the business. Data is at the heart of the organisation and forms a key partof their future plans. **The Opportunity** As a Senior Data Engineer you will be...

  • Data Engineering Team Lead

    3 weeks ago

    London, United Kingdom Areti Group Full time

    Data Team Lead - GIS/Geospatial Data Location: London Work Model: Hybrid, 2 or 3 days a week Salary: ~90k + perks and benefits Areti Group have partnered with a powerhouse in the Data and Product space that specializes in Location Intelligence and Spatial Data, who are seeking a Data Delivery Lead/Head of Data Delivery to join a newly formed team...

  • Senior Data Engineer

    5 days ago

    London, United Kingdom Data Idols Full time

    **Senior Data Engineer** If you're a Senior Data Engineer seeking a meaningful new contract opportunity, look no further. We are working with a fast-growing digital HealthTech company, saving the NHS millions per year, and are radically transforming the delivery of health servicesto make healthcare work for patients, doctors and other healthcare...

  • Senior Data Engineer

    2 days ago

    London, United Kingdom Data Idols Full time

    **Senior Data Engineer** Data Idols are excited to be working with a well-funded startup who are moving into their scale-up phase. We are working to recruit the first Data Engineer, you will be involved in architecting and building their greenfield data platform. In this excitingand dynamic role, you will be involved from the very beginning, selecting the...

  • Senior Data Engineer

    5 days ago

    London, United Kingdom Data Idols Full time

    **Senior Data Engineer** Data Idols are working with a very exciting FinTech who are looking for a Senior Data Engineer to join their Data Team. This role will initially be remote but as restrictions ease the role will be based in their central London office. **THE ROLE** Within this role as a Senior Data Engineer you will be working to help build a...

  • Senior Engineer, Data Engineering

    3 weeks ago

    London, United Kingdom Data Idols Full time

    Senior Data Engineer Location: London, 3/2 days in the office per week Data Idols are currently working with a company that focus on the european marketplace for transport who are looking to make the future greener! Within this role as a Senior Data Engineer, you will be monitoring, designing and implementing a datawarehouse, whilst working with...

  • Senior Engineer, Data Engineering

    Found in: Jooble UK C2 - 3 weeks ago

    London, United Kingdom Data Idols Full time

    Senior Data Engineer Location: London, 3/2 days in the office per week Data Idols are currently working with a company that focus on the european marketplace for transport who are looking to make the future greener! Within this role as a Senior Data Engineer, you will be monitoring, designing and implementing a datawarehouse, whilst working with...

  • Data Engineer

    2 days ago

    City of London, United Kingdom Data Idols Full time

    **Data Engineer - Biotechnology** Data Idols are working with an exciting and innovative biotechnology startup who are working to assist biopharma teams to find better medicines and drugs quicker and more efficiently. **The Opportunity** This is an opportunity for a Data Engineer to join a small team and have a big impact. As a Data Engineer you will...