Lead Data Engineer

2 weeks ago


Newcastle upon Tyne, United Kingdom James Adams Full time

About the Client: Our client are a tech for good business dedicated to promoting transparency in the corporate world they are seeking a highly skilled and experienced Lead Data Engineer.


You will be the first hire in the Data Engineering team allowing you to take the drivers seat regarding the direction taken by the data team to follow.


Key Responsibilities:

  • Platform Development: Design, develop, maintain, test, and optimise a scalable and reliable data platform for collecting, processing, and transforming large volumes of company data from various sources.
  • Collaboration: Work with cross-functional teams to understand data platform requirements and deliver solutions that meet business needs.
  • Optimisation: Enhance data storage and retrieval systems for high performance and data integrity.
  • Quality Assurance: Implement and maintain a data quality and observability framework to ensure data accuracy, completeness, and consistency.
  • Documentation: Develop and maintain comprehensive documentation for data processes, schemas, and data dictionaries.
  • Process Improvement: Identify and implement enhancements to improve data pipeline efficiency and reliability.
  • Cloud Management: Actively manage cloud-based data services to balance performance and cost.
  • Mentorship: Mentor and guide junior data engineers, fostering a culture of continuous learning and development.
  • Innovation: Stay current with the latest data engineering technologies, tools, and best practices.


Required Technical Experience and Skillset:

  • Educational Background: Bachelor’s degree in Computer Science, Engineering, or a related field; Master’s degree is a plus.
  • Experience: 8+ years in data engineering or a similar role.


Technical Skills:

  • Advanced SQL skills and relational database management.
  • Proficiency in at least two object-oriented programming languages (Python, Scala, Java).
  • Deep familiarity with a cloud-native, modern data engineering technology stack compatible with AWS.
  • Experience with web scraping and other data ingestion methods and tools.
  • Knowledge of distributed computing frameworks (Hadoop, Spark, Hive, Presto).
  • Experience with data orchestration tools (Airflow, Orchestra, Azkaban).
  • Expertise in cloud data warehousing and core data modelling concepts.
  • Proficiency in version control systems (Git) and experience with CI/CD pipelines.
  • Project Management: Strong organisational and project management skills.
  • Leadership: Proven ability to lead a data engineering platform implementation from scratch, working independently and as part of a team.


Preferred Experience and Skillset:

  • Certifications: Data Engineering Certification is a plus.
  • Database Design: Expertise in SQL and NoSQL databases (PostgreSQL, Cassandra, Elasticsearch).
  • Stream Processing: Familiarity with stream processing technologies.
  • Data Architecture: Deep understanding of data architecture considerations (data lakes, data warehouses, data lakehouses).
  • AI/ML Integration: Experience deploying AI or machine learning models into production.
  • Domain Knowledge: Experience with legal entity, UBO, or other datasets relevant to KYB/AML use cases.


Personal Attributes:

  • Strong team fit and aptitude, with a focus on collaboration and problem-solving.
  • Ability to trade ideas with peers to reach optimal solutions.
  • Practical approach to incremental delivery of objectives.
  • Capacity to balance perfection with the need for progress.
  • Enthusiasm for solving complex problems at scale.



  • Newcastle upon Tyne, United Kingdom Rain Data Ltd Full time

    This role will be a full time position and require being office based initially during training, looking to then move to a hybrid role.   In this role you will be responsible for supervising, organising and motivating team members who are in data technician roles including apprentices.   As a team lead you will be the contact point for all data...

  • Lead Data Engineer

    3 days ago


    Newcastle upon Tyne, United Kingdom DWP Digital Full time

    DWP. Digital with purpose. Are you a leader in data engineer with the experience and drive to lead a team of talented Data Engineers to success? Are you passionate about building and supporting data capabilities that are essential to DWP services? Do you want to be involved in projects that have genuine impact on millions on UK citizens? DWP Digital is...

  • Lead Data Engineer

    4 days ago


    Newcastle upon Tyne, United Kingdom DWP Digital Full time

    DWP. Digital with purpose.Are you a leader in data engineer with the experience and drive to lead a team of talented Data Engineers to success?Are you passionate about building and supporting data capabilities that are essential to DWP services?Do you want to be involved in projects that have genuine impact on millions on UK citizens?DWP Digital is...

  • Lead Data Engineer

    5 days ago


    Newcastle upon Tyne, United Kingdom Thoughtworks Inc. Full time

    Lead data engineers at Thoughtworks develop modern data architecture approaches to meet key business objectives and provide end-to-end data solutions. They might spend a few weeks with a new client on a deep technical review or a complete organizational review, helping them to understand the potential that data brings to solve their most pressing problems....

  • SAP DATA Team Lead

    7 days ago


    Newcastle upon Tyne, United Kingdom NTT DATA Business Solutions Full time

    Data Management – SAP Data Team Lead NTT DATA Business Solutions – NTT DATA Business Solutions is part of the NTT DATA Corporate Group a top 10 global IT services provider, headquartered in Tokyo, operating in more than 50 countries. We combine a global reach with local intimacy to provide premier professional SAP services from deep industry expertise...


  • Newcastle upon Tyne, United Kingdom Rain Data Ltd Full time

    This role will be a full time position and require being office based initially during training, looking to then move to a hybrid role.   In this role you will be responsible for supervising, organising and motivating team members who are in data technician roles including apprentices.   As a team lead you will be the contact point for all data...

  • Data Engineer

    2 weeks ago


    Newcastle upon Tyne, United Kingdom Xpertise Recruitment Full time

    Xpertise is seeking a series of Data Engineers with cloud experience to join a fledgling team in Birmingham. As part of our client's growing engineering division, you will play a pivotal role in leading the data engineering capabilities, working closely with Platform Engineers, Developers, and Analysts.Key details:Salary: £(phone number removed)...

  • Senior planning lead

    1 month ago


    Newcastle upon Tyne, United Kingdom Lead Candidate Full time

    Wherever you operate across the global drug development and manufacturing supply chain, we exist for you. We have walked in your shoes, know the industry and the frustrations never ending talent needs bring to your world. That’s where we help. Specialties Life Sciences, Pharmaceuticals, Biotechnology, Talent acquisition, Skills, Bio Tech Outsourcing,...

  • Data Engineer

    6 days ago


    Newcastle upon Tyne, United Kingdom BJSS Full time

    About UsnWe’re an award-winning innovative tech consultancy - a team of creative problem solvers. Since 1993 we’ve been finding better, more sustainable ways to solve complex technology problems for some of the world’s leading organisations and delivered solutions that millions of people use every day.In the last 30 years we won several awards,...

  • Data Engineer

    4 days ago


    Newcastle upon Tyne, United Kingdom BJSS Full time

    Job Description About Us We’re an award-winning innovative tech consultancy - a team of creative problem solvers. Since 1993 we’ve been finding better, more sustainable ways to solve complex technology problems for some of the world’s leading organisations and delivered solutions that millions of people use every day. In the last 30 years we won...

  • Data Platform Lead

    1 month ago


    Newcastle upon Tyne, United Kingdom Greggs Full time

    About the role - This is a full-time role, however flexibility in this will be considered - We know that having a work-life balance is important, so we offer our colleagues as much flexibility as possible in line with the needs of their role - The base location for this role is Greggs Innovate House, Newcastle upon Tyne, NE12 8EW What you'll do Reporting...

  • Data Engineer

    5 days ago


    Newcastle Upon Tyne, United Kingdom Harnham Full time

    **Data Engineer** **Durham 1x week** **Up to £50,000 per annum** **THE COMPANY** This company is one of the largest electrical wholesalers in the UK, which has now branched out to the USA, and is turning over $3 billion per year! They work with businesses and independent customers to provide the highest service and pride themselves ontheir fantastic...

  • Data Engineer

    18 hours ago


    Newcastle upon Tyne, United Kingdom Xcede Full time

    Data Engineer (AWS)£70,000 Salary + BonusLondon office 2-3 days per week.OVERVIEWA successful AI scale up in the creative space are hiring for a Data Engineer (AWS) to join their well established Data team, spearheading some of their key Data Engineering projects in AWS and mentoring junior team members. Your responsibilities as a Data Engineer will include...


  • Newcastle upon Tyne, United Kingdom Thoughtworks Inc. Full time

    Principal data engineers at Thoughtworks, are strategic leaders who spearhead data engineering initiatives, tackle complex business challenges and uncover transformative insights. They possess a deep understanding of a client's business ecosystem and partner with executives to align technology strategies with business objectives. By contextualizing...


  • Newcastle upon Tyne, United Kingdom Thoughtworks Inc. Full time

    Principal data engineers at Thoughtworks, are strategic leaders who spearhead data engineering initiatives, tackle complex business challenges and uncover transformative insights. They possess a deep understanding of a client's business ecosystem and partner with executives to align technology strategies with business objectives. By contextualizing...


  • Newcastle upon Tyne, Newcastle upon Tyne, United Kingdom BBC Full time

    Job BandPackage description Band: D Contract Permanent Salary : £65,000 to £85,000 depending on level of experience Location: Salford, and Newcastle Our comprehensive benefits package includes: An employer pension contribution of up to 10% 26 days' annual leave (based on full time hours) + bank holidays and the option to buy/sell additional days...

  • Data Engineer

    5 days ago


    Newcastle Upon Tyne, United Kingdom Steria Recruitment Full time

    **Job Title: Data Engineer** **Location: Hybrid/Newcastle** **Duration: Permanent** **Role**: As a Data Engineer, you'll be joining the Digital Policing function and play a key role in providing effective data solutions but will equally understand the impact of a data failure to the business and support the user through the experience to its...

  • Data Engineer

    1 week ago


    Newcastle Upon Tyne, United Kingdom Steria Recruitment Full time

    **Job Title: Senior Data Engineer** **Location: Hybrid/Newcastle** **Duration: Permanent** **Role**: As the Senior Data Engineer, you'll be joining the Digital Policing function and play a key role in providing effective data solutions but will equally understand the impact of a data failure to the business and support the user through the experience toits...


  • Newcastle upon Tyne, United Kingdom Abs Data Full time

    Principal Electrical / ICA Engineer – Newcastle In today’s world it’s important to work for a company that has clear purpose, giving back to communities and supporting what is truly important in the world. To find out more about our Water business click on the following link and discover what awaits you at WSP: WSP is currently recruiting for an...


  • Newcastle upon Tyne, United Kingdom CK Group- Science, Clinical and Technical Full time

    CK Group are recruiting for a highly motivated Laboratory Automation Lead to join a leading biotechnological company in Newcastle. A newly formed biotechnology organisation focused on building a biological and digital platform for biological assets. They have state-of-the-art microbial engineering facilities (including automation equipment), which the...