Data Lakes Specialist

6 days ago


London, Greater London, United Kingdom N Consulting Limited Full time

**Job Title:** Data Lakes Specialist

**Location:** London, UK

We are seeking an experienced Data Lakes Specialist to join our team at N Consulting Limited. As a key member of our Big Data team, you will be responsible for designing, building, and managing scalable data lakes using the Hadoop ecosystem.

**Key Responsibilities:

  1. Design and implement data ingestion pipelines and workflows for structured, semi-structured, and unstructured data.
  2. Develop and maintain Big Data solutions using HDFS, Hive, HBase, Spark, Pig, MapReduce, etc.
  3. Collaborate with data engineers, analysts, and scientists to provide robust and efficient data access solutions.
  4. Monitor and troubleshoot data pipelines and applications to ensure reliability and accuracy.
  5. Implement data security, governance, and compliance practices across the data lake and Hadoop systems.

**Required Skills and Qualifications:

  • Proven experience with Hadoop ecosystems, including HDFS, YARN, Hive, HBase, MapReduce, and Spark.
  • Expertise in Data Lake architectures and principles.
  • Proficiency in programming languages such as Python, Java, or Scala for Big Data processing.
  • Familiarity with ETL tools, data ingestion frameworks, and workflow schedulers (e.g., Apache NiFi, Airflow).
  • Strong knowledge of cloud platforms such as AWS (S3, EMR, Glue), Azure (Data Lake Storage, Synapse), or Google Cloud (BigQuery, Dataflow).
  • Familiarity with query languages like SQL, HiveQL, or Presto.

**Estimated Salary:** £80,000 - £110,000 per annum, depending on experience.



  • London, Greater London, United Kingdom Adecco UK LTD Full time

    Job Summary: Cloud Data Lake SpecialistSalary: £70,000 - £90,000 per annumWe are seeking a highly skilled Cloud Data Lake Specialist to join our team in Farringdon or Welwyn Garden City. As a key member of the Cyber Analytics team, you will be responsible for building data models, ETLs, and creating insights that drive cybersecurity strategy and...

  • Data Lake Engineer

    2 weeks ago


    London, Greater London, United Kingdom Conexus Full time

    Azure Data Architect Job OpportunityWe are looking for a skilled Azure Data Architect to join our team in London. This is a 6-month contract, with the possibility of further extension. As a member of our team, you will be responsible for designing and developing cloud-based data architectures using Microsoft Azure. Key responsibilities include developing and...

  • Data Lake Architect

    3 weeks ago


    London, Greater London, United Kingdom Tesco Partners Full time

    Job OverviewTesco Partners is seeking a highly skilled Staff Analytics Engineer to join their Cyber Analytics team. This role requires a strong background in data engineering, analytics, and cybersecurity.About the RoleThis is an exciting opportunity for a talented individual to design, develop, and deploy scalable data lake and data warehouse solutions. The...


  • London, Greater London, United Kingdom Station Full time

    Job Title: Central London Data Lake ManagerOur client is seeking an experienced Central London Data Lake Manager to lead the development of their centralised data storage solution. As part of this exciting project, you will be responsible for designing, implementing, and managing the Data Lake, ensuring it meets the organisation's needs and adheres to...


  • London, Greater London, United Kingdom Data Science Festival Full time

    Senior Data Engineer PositionData Science Festival is currently seeking an experienced Senior Data Engineer to lead our data platform efforts. As a key contributor to our data-driven decision-making process, you will design, implement, and maintain scalable data infrastructure using Azure Services.This role requires expertise in Azure Data Lake Storage,...


  • London, Greater London, United Kingdom Data Idols Full time £80,000 - £90,000

    The Perfect CandidateAs a Senior Data Platform Engineer, you'll have:Expertise in Azure Data Lake Storage, Azure Compute, and Azure Databricks.A strong understanding of DevOps practices, using tools like Azure DevOps, Jenkins, or GitHub Actions.Solid knowledge of security best practices on Azure, including identity and access management, network security,...


  • London, Greater London, United Kingdom Data Science Festival Full time

    Job OverviewSenior Data Platform EngineerAs a Senior Data Platform Engineer at our prestigious retailer, you'll be part of a talented, collaborative team that's evolving into a digital-first, data-led organisation. Your expertise will shape the data landscape and ensure the delivery of high-quality data solutions.You'll work closely with cross-functional...


  • London, Greater London, United Kingdom Data Intellect Full time

    Lead Data Engineer & ArchitectWe are looking for a highly skilled Cloud Data Architect Lead to lead our Databricks practice and drive innovation in data-driven solutions. This role requires experience in designing and deploying large-scale data pipelines, ETL processes, and data lakes using Databricks.The successful candidate will have a solid understanding...


  • London, Greater London, United Kingdom Data Careers Full time

    Job Description: Data Engineer SC Cleared We are looking to recruit a Data Engineer with SC Clearance for a leading software and solutions organisation. The successful candidate will design, develop and maintain robust data infrastructure and ETL pipelines that support the data needs of projects, services and customers. Key Responsibilities: Build data...


  • London, Greater London, United Kingdom Fruition IT Full time £65,000

    **Job Title:** Data Transformation SpecialistFruition IT is seeking a skilled Data Transformation Specialist to drive their digital transformation efforts. This full-time, permanent position offers the opportunity to shape the company's data strategy, design scalable infrastructure, and create a unified data lake that will serve as the backbone for reporting...


  • London, Greater London, United Kingdom Epsilon Data Management, LLC Full time

    Job Description: Epsilon Data Management, LLC is seeking a talented Data Management Systems Specialist to join our team. As a key member of our operations team, you will be responsible for designing, implementing, and maintaining our data management systems, ensuring they meet the needs of our clients and stakeholders. This role requires strong technical...


  • London, Greater London, United Kingdom Data Science Festival Full time

    We are seeking a seasoned Senior Data Platform Engineer to join our team at Data Science Festival. As a key member of our data engineering team, you will play a vital role in shaping the data landscape and contributing directly to data-driven decision-making.In this role, you will design, implement, and maintain robust data platforms that underpin our...

  • Cloud Data Architect

    2 weeks ago


    London, Greater London, United Kingdom Data Idols Full time

    Job OverviewData Idols is seeking a skilled Senior Data Platform Engineer to join its team. As a key member of the data platform engineering group, you will play a vital role in designing and implementing scalable, reliable, and secure data infrastructure using Azure Services.Responsibilities- Designing and implementing scalable, reliable, and secure data...


  • London, Greater London, United Kingdom Data Idols Full time £80,000 - £90,000

    Data Engineering OpportunityData Idols is seeking a skilled Senior Data Platform Engineer to join our team. As a key member of our engineering team, you will be responsible for designing, implementing, and maintaining a robust data platform using Azure Services.This is an exciting opportunity to work with cutting-edge cloud-based technologies and collaborate...


  • London, Greater London, United Kingdom Data Science Festival Full time

    Job DescriptionWe are seeking a talented Senior Data Platform Engineer to play a pivotal role in designing, implementing, and maintaining our robust data platform infrastructure using Azure Services. This individual will lead CI/CD deployments for data infrastructure, ensuring seamless integrations with other business applications. Key responsibilities...


  • London, Greater London, United Kingdom Data Science Festival Full time

    Product Data StrategistWork alongside the Data Science and Data Engineering team as a Product Data Strategist, focusing on developing and maintaining the roadmap for products.This strategic role involves fulfilling agile product management responsibilities for data products, fostering collaboration with stakeholders, and driving data product utilization.Main...


  • London, Greater London, United Kingdom Lorien Full time

    Senior Asset Data Specialist Wanted by LorienA 6-month contract opportunity is available for an experienced data specialist to work on a high-profile project in the UK.About the Role:Essential experience in Financial Services and Asset Management.The ideal candidate will be familiar with industry-standard data modelling tools, such as Erwin.Datasets related...


  • London, Greater London, United Kingdom Harnham Full time £55,000 - £65,000

    Data Engineer SpecialistSalary: £55,000-65,000 per annumA big data developer is required to join the product engineering organisation of a UK-based music company. The successful candidate will be responsible for building a new data lake in AWS and creating data pipelines using Python.About the CompanyThe company is one of the largest in the UK, helping...

  • Data Roadmap Manager

    2 weeks ago


    London, Greater London, United Kingdom Data Science Festival Full time

    Data Roadmap ManagerWe are looking for a skilled Data Roadmap Manager to join our team and lead the development and maintenance of product roadmaps, aligning with business objectives.This role requires effective collaboration with the Data Science and Data Engineering team, as well as stakeholders, to drive data product adoption and utilization.Main...


  • London, Greater London, United Kingdom Data Idols Full time

    About the RoleData Idols is a leading organisation in the field of data-driven decision-making. We are currently seeking a Senior Data Platform Engineer to join our team and contribute to shaping the data landscape. In this role, you will work alongside a talented, collaborative team to build, optimise, and secure scalable data infrastructure.Key...