Lead Data Engineer

4 weeks ago

UK, UK, United Kingdom James Adams Full time

About the Client: Our client are a tech for good business dedicated to promoting transparency in the corporate world they are seeking a highly skilled and experienced Lead Data Engineer.


You will be the first hire in the Data Engineering team allowing you to take the drivers seat regarding the direction taken by the data team to follow.


Key Responsibilities:

  • Platform Development: Design, develop, maintain, test, and optimise a scalable and reliable data platform for collecting, processing, and transforming large volumes of company data from various sources.
  • Collaboration: Work with cross-functional teams to understand data platform requirements and deliver solutions that meet business needs.
  • Optimisation: Enhance data storage and retrieval systems for high performance and data integrity.
  • Quality Assurance: Implement and maintain a data quality and observability framework to ensure data accuracy, completeness, and consistency.
  • Documentation: Develop and maintain comprehensive documentation for data processes, schemas, and data dictionaries.
  • Process Improvement: Identify and implement enhancements to improve data pipeline efficiency and reliability.
  • Cloud Management: Actively manage cloud-based data services to balance performance and cost.
  • Mentorship: Mentor and guide junior data engineers, fostering a culture of continuous learning and development.
  • Innovation: Stay current with the latest data engineering technologies, tools, and best practices.


Required Technical Experience and Skillset:

  • Educational Background: Bachelor’s degree in Computer Science, Engineering, or a related field; Master’s degree is a plus.
  • Experience: 8+ years in data engineering or a similar role.


Technical Skills:

  • Advanced SQL skills and relational database management.
  • Proficiency in at least two object-oriented programming languages (Python, Scala, Java).
  • Deep familiarity with a cloud-native, modern data engineering technology stack compatible with AWS.
  • Experience with web scraping and other data ingestion methods and tools.
  • Knowledge of distributed computing frameworks (Hadoop, Spark, Hive, Presto).
  • Experience with data orchestration tools (Airflow, Orchestra, Azkaban).
  • Expertise in cloud data warehousing and core data modelling concepts.
  • Proficiency in version control systems (Git) and experience with CI/CD pipelines.
  • Project Management: Strong organisational and project management skills.
  • Leadership: Proven ability to lead a data engineering platform implementation from scratch, working independently and as part of a team.


Preferred Experience and Skillset:

  • Certifications: Data Engineering Certification is a plus.
  • Database Design: Expertise in SQL and NoSQL databases (PostgreSQL, Cassandra, Elasticsearch).
  • Stream Processing: Familiarity with stream processing technologies.
  • Data Architecture: Deep understanding of data architecture considerations (data lakes, data warehouses, data lakehouses).
  • AI/ML Integration: Experience deploying AI or machine learning models into production.
  • Domain Knowledge: Experience with legal entity, UBO, or other datasets relevant to KYB/AML use cases.


Personal Attributes:

  • Strong team fit and aptitude, with a focus on collaboration and problem-solving.
  • Ability to trade ideas with peers to reach optimal solutions.
  • Practical approach to incremental delivery of objectives.
  • Capacity to balance perfection with the need for progress.
  • Enthusiasm for solving complex problems at scale.

  • Data Engineering Lead

    4 weeks ago

    UK, UK, United Kingdom ON Data Staffing Full time

    We are seeking a dynamic and experienced Databricks professional to join our client, a growing startup consultancy, as a Lead Data Engineer.As Lead Data Engineer, you will play a pivotal role in driving the success of our client's Databricks practice through your expertise in presales, strong technical acumen, consulting skills, and effective team...

  • Senior Data Engineer

    6 days ago

    UK, UK, United Kingdom ON Data Staffing Full time

    Our client, a Data & AI firm in London is actively seeking a Senior Data Engineer to play a pivotal role in delivering innovative big data solutions to their end clients.This client-facing role requires a unique blend of technical prowess in Databricks/Snowflake and exceptional communication skills to understand and address clients' diverse data...

  • Data Engineering Manager

    4 weeks ago

    UK, UK, United Kingdom ON Data Staffing Full time

    We are seeking a dynamic and experienced Data Engnieering professional to join our client, a growing consultancy, as a Data Engineering Manager.As Data Engineering Manager, you will play a pivotal role in driving the success of our client's Data Engineering practice through your expertise in presales, strong technical acumen, consulting skills, and...

  • Data Engineering Manager

    3 weeks ago

    UK, UK, United Kingdom ON Data Staffing Full time

    We are seeking a dynamic and experienced Data Engnieering professional to join our client, a growing consultancy, as a Data Engineering Manager.As Data Engineering Manager, you will play a pivotal role in driving the success of our client's Data Engineering practice through your expertise in presales, strong technical acumen, consulting skills, and...

  • Senior Data Engineer

    4 weeks ago

    UK, UK, United Kingdom ON Data Staffing Full time

    Our client, a boutique Data & AI Consultancy in London is actively seeking multiple Senior Data Engineer Consultants to play a pivotal role in delivering innovative Databricks technical solutions to their end clients.This client-facing role requires a unique blend of technical prowess in Databricks and exceptional communication skills to understand and...

  • Senior Data Engineer

    7 days ago

    London, UK, UK, United Kingdom Data Idols Full time

    Senior Data EngineerSalary: up to £85,000Location: London (Hybrid)Data Idols are working with an exciting scale-up revolutionising delivery logistics and moving items across the UK with efficiency and ease all whilst cutting emissions. Trusted by millions of customers already they are expanding their data team to manage increased volumes of data and demand....

  • Lead Data Engineer

    4 weeks ago

    UK, UK, United Kingdom Ventula Consulting Full time

    Lead Data Engineer - Snowflake / DBT / Python - ecommerce - £95kLeading eCommerce client are now searching for a Lead Data Engineer to play a pivotal role in delivering their innovative data strategy.This role will shape our client’s data function, delivering end to end data requirements and designing data pipelines/warehouses to ensure data is...

  • Lead Data Engineer

    3 weeks ago

    UK, UK, United Kingdom James Adams Full time

    About the Client: Our client are a tech for good business dedicated to promoting transparency in the corporate world they are seeking a highly skilled and experienced Lead Data Engineer. You will be the first hire in the Data Engineering team allowing you to take the drivers seat regarding the direction taken by the data team to follow.Key...

  • Lead Data Engineer

    4 days ago

    UK, UK, United Kingdom James Adams Full time

    About the Client: Our client are a tech for good business dedicated to promoting transparency in the corporate world they are seeking a highly skilled and experienced Lead Data Engineer.You will be the first hire in the Data Engineering team allowing you to take the drivers seat regarding the direction taken by the data team to follow. You will report into...

  • Lead Data Engineer

    4 weeks ago

    London, UK, UK, United Kingdom DATAHEAD Full time

    ✨ Lead Data Engineer Opportunity | ️ Azure Cloud | ️ Lloyds Insurer ✨A leading insurance business collaborates with DATAHEAD to find a talented Senior Data Engineer to join their Data Team . This is an incredible chance to work with technology and data-driven Lloyd's insurer whose size offers agility and allows you to help shape its future !...

  • Lead Data Engineer

    3 weeks ago

    London, UK, UK, United Kingdom DATAHEAD Full time

    ✨ Lead Data Engineer Opportunity | ️ Azure Cloud | ️ Lloyds Insurer ✨A leading insurance business collaborates with DATAHEAD to find a talented Senior Data Engineer to join their Data Team . This is an incredible chance to work with technology and data-driven Lloyd's insurer whose size offers agility and allows you to help shape its future !...

  • Lead Data Engineer

    4 weeks ago

    London, UK, UK, United Kingdom GCS Full time

    Lead Data EngineerPermanent – HybridSalary up to £70,186 p/year, 35 hours p/weekJoin our dynamic public sector client based in Central London, where you will spearhead the technical design and implementation of transformative data integration projects. They are committed to leveraging cutting-edge technologies and cloud components to enhance the data...

  • Lead Data Engineer

    4 weeks ago

    London, UK, UK, United Kingdom Arrows Full time

    Lead Data EngineerI'm currently working on a dynamic opportunity which is looking for a Lead Data Engineer to join and lead thier team! You'll play a pivotal role in bridging engineering best practices with thier data domain. Thier team is committed to transforming data into actionable insights, aiding decision-making processes and enhancing our...

  • Lead Data Engineer

    3 weeks ago

    London, UK, UK, United Kingdom Arrows Full time

    Lead Data EngineerI'm currently working on a dynamic opportunity which is looking for a Lead Data Engineer to join and lead thier team! You'll play a pivotal role in bridging engineering best practices with thier data domain. Thier team is committed to transforming data into actionable insights, aiding decision-making processes and enhancing our...

  • Lead Data Engineer with Palantir Foundry

    4 weeks ago

    UK, UK, United Kingdom Saransh Inc Full time

    Lead Data Engineer with Palantir FoundryRemote (UK & Europe)ContractNOTE:- The Resource Needs to work In European Time ZoneDescription:Total experience of 7+ years in Data Engineering and must have at least 3 years of experience in Palantir Foundry.Strong experience with Palantir Data Engineering features such as, Code Repo, Code Workbook, Pipeline Build,...

  • Lead Data Engineer with Palantir Foundry

    3 weeks ago

    UK, UK, United Kingdom Saransh Inc Full time

    Lead Data Engineer with Palantir FoundryRemote (UK & Europe)ContractNOTE:- The Resource Needs to work In European Time ZoneDescription:Total experience of 7+ years in Data Engineering and must have at least 3 years of experience in Palantir Foundry.Strong experience with Palantir Data Engineering features such as, Code Repo, Code Workbook, Pipeline Build,...

  • Lead Data Engineer

    7 days ago

    London, UK, UK, United Kingdom Harnham Full time

    LEAD DATA ENGINEERLONDON, CURRENTLY FLEXIBLE / REMOTE / HOMEWORKINGUP TO £100,000 + BENEFITSAs a Lead Data Engineer, you will be inheriting a Snowflake data platform for a successful Ecommerce company that has disrupted the market with their product line. You will be leading a junior squad so will remain hands-on to further develop the platform and team...

  • Lead Data Architect

    4 weeks ago

    UK, UK, United Kingdom Intermedia Cloud Communications Full time

    LEAD DATA ARCHITECTAll candidates must be located in the UKAre you looking for a company where YOUR VOICE is heard? Where you can MAKE A DIFFERENCE? Do you THRIVE in a FAST-PACED work environment? Do you wake every morning EXCITED to work with GREAT PEOPLE and create SUCCESS TOGETHER? Then Intermedia is the place for you.Intermedia has established itself as...

  • Lead Data Architect

    3 weeks ago

    UK, UK, United Kingdom Intermedia Cloud Communications Full time

    LEAD DATA ARCHITECTAll candidates must be located in the UKAre you looking for a company where YOUR VOICE is heard? Where you can MAKE A DIFFERENCE? Do you THRIVE in a FAST-PACED work environment? Do you wake every morning EXCITED to work with GREAT PEOPLE and create SUCCESS TOGETHER? Then Intermedia is the place for you.Intermedia has established itself as...

  • Technical Lead

    7 days ago

    London, UK, UK, United Kingdom Harnham Full time

    Data Engineer - Tech LeadFintechLondon£110,000As a Tech Lead, you will be leading and managing a greenfield implementation for a disruptive Fintech. The data platform will be used to facilitate the launch of a new product. THE COMPANY:This company has gone through a period of hyper-growth over the last 3 years. They specialise in the credit card space and...