Senior Data Engineer

1 week ago

London, Greater London, United Kingdom Salt Full time
Job Description

Job Summary

Salt is seeking a highly skilled Cloud Data Engineer to join our team. As a Cloud Data Engineer, you will be responsible for designing, building, and maintaining large-scale data systems on AWS.

Key Responsibilities

  • Cloud and Big Data Administration:
  • Manage and optimize big data environments on AWS, ensuring efficient administration and maintenance.
  • Data Engineering and Architecture:
  • Design and implement data lakehouses versus data fabric architectures, ensuring data integrity and accessibility.
  • Spark Management:
  • Develop and maintain Spark jobs, with a focus on auditing, monitoring, and instrumentation to ensure reliability and performance.
  • Infrastructure as Code (IaC):
  • Utilize Terraform for infrastructure provisioning and management, ensuring scalable and secure environments.
  • Data Storage and Management:
  • Work with data storage formats like Parquet and Hudi to optimize data storage and retrieval.
  • Security and Compliance:
  • Implement and manage IAM policies for secure data access and management across AWS services.
  • Collaboration and Continuous Improvement:
  • Collaborate with cross-functional teams to develop and deploy data solutions.
  • Continuously improve data engineering practices, leveraging new tools and techniques to enhance performance and efficiency.

Requirements

  • Cloud and Big Data:
  • Extensive experience in cloud DevOps and big data administration on AWS.
  • Proficiency in AWS Lake Formation, with a strong understanding of data lakehouse versus data fabric concepts.
  • Programming and Data Engineering:
  • Expertise in Python for data processing and automation.
  • Deep knowledge of Spark internals (Spark 3+ features), including architecture, events, system metrics, AQE, and job-level resource management.
  • Tech Stack:
  • Strong hands-on experience with Terraform for infrastructure management.
  • Experience with data formats such as Hudi and Parquet.
  • Familiarity with AWS Glue, HIVE, and IAM for data management and security.

Good to Have

  • Additional Tools and Technologies:
  • Knowledge of Iceberg and its application in data storage and management.
  • Experience with Airflow for workflow automation.
  • Familiarity with Terragrunt for managing Terraform configurations.
  • Understanding of DynamoDB and its integration within data environments.

  • Senior Data Engineering Project Manager

    7 days ago

    London, Greater London, United Kingdom NTT DATA Full time

    About the RoleWe are seeking an experienced Senior Data Engineering Project Manager to join our team at NTT DATA. As a key member of our Data Engineering service line, you will be responsible for overseeing Data Engineering projects and programmes across our Data transformation portfolio.Key ResponsibilitiesLead highly-skilled technical teams in a...

  • Senior Data Engineering Project Manager

    6 days ago

    London, Greater London, United Kingdom NTT DATA Full time

    About the RoleWe are seeking an experienced Senior Data Engineering Project Manager to join our team at NTT DATA. As a key member of our Data Engineering service line, you will be responsible for overseeing Data Engineering projects and programmes across our Data transformation portfolio.Key ResponsibilitiesLead highly-skilled technical teams in a...

  • Senior Data Engineering Manager

    2 days ago

    London, Greater London, United Kingdom ON Data Staffing Full time

    About the RoleWe are seeking a highly skilled and experienced Senior Consulting Manager - Data Engineering to lead and grow our data engineering practice in the UK.Key ResponsibilitiesLead and manage the Data Engineering practice, delivering projects across cloud data platforms such as Snowflake and Databricks.Provide strategic direction on data...

  • Senior Data Engineering Manager

    2 days ago

    London, Greater London, United Kingdom ON Data Staffing Full time

    About the RoleWe are seeking a highly skilled and experienced Senior Consulting Manager - Data Engineering to lead and grow our data engineering practice in the UK.Key ResponsibilitiesLead and manage the Data Engineering practice, delivering projects across cloud data platforms such as Snowflake and Databricks.Provide strategic direction on data...

  • Senior Blockchain Engineer

    4 days ago

    London, Greater London, United Kingdom Lawton Senior Living Full time

    About the RoleWe are seeking a highly skilled Senior Blockchain Engineer to join our team at Lawton Senior Living. As a Senior Blockchain Engineer, you will play a critical role in designing, building, and maintaining large-scale data systems that provide foundational data for our organization's products and services.Key ResponsibilitiesDesign and develop...

  • Senior Blockchain Engineer

    4 days ago

    London, Greater London, United Kingdom Lawton Senior Living Full time

    About the RoleWe are seeking a highly skilled Senior Blockchain Engineer to join our team at Lawton Senior Living. As a Senior Blockchain Engineer, you will play a critical role in designing, building, and maintaining large-scale data systems that provide foundational data for our organization's products and services.Key ResponsibilitiesDesign and develop...

  • Senior Data Engineer

    8 hours ago

    London, Greater London, United Kingdom Tribal Tech - The Digital, Data & AI Specialists Full time £70,000 - £90,000

    Senior Data Engineer OpportunityTribal Tech - The Digital, Data & AI Specialists is seeking an experienced Senior Data Engineer to join their innovative team. This role offers a unique opportunity to shape the future of data-driven decision-making within a rapidly growing organization.Key Responsibilities:Design, implement, and maintain robust data pipelines...

  • Senior Data Engineer

    6 hours ago

    London, Greater London, United Kingdom Tribal Tech - The Digital, Data & AI Specialists Full time £70,000 - £90,000

    Senior Data Engineer OpportunityTribal Tech - The Digital, Data & AI Specialists is seeking an experienced Senior Data Engineer to join their innovative team. This role offers a unique opportunity to shape the future of data-driven decision-making within a rapidly growing organization.Key Responsibilities:Design, implement, and maintain robust data pipelines...

  • Data Engineer I

    5 days ago

    London, Greater London, United Kingdom Tampa Gardens Senior Living Full time

    Senior Data EngineerAbout the Role:We are seeking a highly skilled Senior Data Engineer to join our team at Tampa Gardens Senior Living. As a Senior Data Engineer, you will play a critical role in designing, implementing, and maintaining high-quality datasets and data pipelines to support business decision-making.Key Responsibilities:Lead the development of...

  • Data Engineer I

    5 days ago

    London, Greater London, United Kingdom Tampa Gardens Senior Living Full time

    Senior Data EngineerAbout the Role:We are seeking a highly skilled Senior Data Engineer to join our team at Tampa Gardens Senior Living. As a Senior Data Engineer, you will play a critical role in designing, implementing, and maintaining high-quality datasets and data pipelines to support business decision-making.Key Responsibilities:Lead the development of...

  • Technical Delivery Lead

    7 days ago

    London, Greater London, United Kingdom NTT DATA Full time

    About the RoleWe are seeking an experienced Senior Data Engineering Project Manager to join our team at NTT DATA. As a key member of our Data Engineering service line, you will be responsible for overseeing Data Engineering projects and programmes across our Data transformation portfolio.Key ResponsibilitiesLead highly-skilled technical teams in a...

  • Technical Delivery Lead

    6 days ago

    London, Greater London, United Kingdom NTT DATA Full time

    About the RoleWe are seeking an experienced Senior Data Engineering Project Manager to join our team at NTT DATA. As a key member of our Data Engineering service line, you will be responsible for overseeing Data Engineering projects and programmes across our Data transformation portfolio.Key ResponsibilitiesLead highly-skilled technical teams in a...

  • Senior Data Engineer

    3 hours ago

    London, Greater London, United Kingdom Harnham - Data & Analytics Recruitment Full time

    About the RoleWe are seeking a highly skilled Senior Data Engineer to join our expanding data team at Harnham - Data & Analytics Recruitment. As a Senior Data Engineer, you will be responsible for building and maintaining our company's Azure platform, working closely with stakeholders to understand data collected, and implementing best coding practices using...

  • Senior Data Engineer

    31 minutes ago

    London, Greater London, United Kingdom Harnham - Data & Analytics Recruitment Full time

    About the RoleWe are seeking a highly skilled Senior Data Engineer to join our expanding data team at Harnham - Data & Analytics Recruitment. As a Senior Data Engineer, you will be responsible for building and maintaining our company's Azure platform, working closely with stakeholders to understand data collected, and implementing best coding practices using...

  • Data Engineering Leader

    6 days ago

    London, Greater London, United Kingdom Harnham - Data & Analytics Recruitment Full time £120,000

    About the RoleWe are partnering with a leading InsurTech company to find a seasoned Data Engineering Leader to join their team. This is an exciting opportunity to grow the data platform and build a high-performing team.Key ResponsibilitiesDevelop and implement a strategy to grow the data team and introduce new technologies to ensure the platform is built...

  • Data Engineering Leader

    6 days ago

    London, Greater London, United Kingdom Harnham - Data & Analytics Recruitment Full time £120,000

    About the RoleWe are partnering with a leading InsurTech company to find a seasoned Data Engineering Leader to join their team. This is an exciting opportunity to grow the data platform and build a high-performing team.Key ResponsibilitiesDevelop and implement a strategy to grow the data team and introduce new technologies to ensure the platform is built...

  • Cloud Data Engineer

    6 days ago

    London, Greater London, United Kingdom SENIOR SPIRIT OF ROSELLE PARK Full time

    About Senior Spirit of Roselle ParkSenior Spirit of Roselle Park is a leading provider of data science solutions, with a team of over 350 data scientists and 1,000 projects since 2006. Our global presence in Paris, London, NY, and HK enables us to lead projects in over 50 countries across various industries, including automotive, financial services, retail,...

  • Senior Data Engineer

    6 days ago

    London, Greater London, United Kingdom SENIOR SPIRIT OF ROSELLE PARK Full time

    About Senior Spirit of Roselle ParkSenior Spirit of Roselle Park is a leading provider of data science solutions, with a team of over 350 data scientists and 1,000 projects since 2006. Our global presence in Paris, London, NY, and HK enables us to lead projects in over 50 countries across various industries, including automotive, financial services, retail,...

  • Senior Data Engineer

    1 week ago

    London, Greater London, United Kingdom SENIOR SPIRIT OF ROSELLE PARK Full time

    About Senior Spirit of Roselle ParkSenior Spirit of Roselle Park is a leading provider of data science solutions, with a team of over 350 data scientists and 1,000 projects since 2006. Our global presence in Paris, London, NY, and HK enables us to lead projects in over 50 countries across various industries, including automotive, financial services, retail,...

  • Cloud Data Engineer

    1 week ago

    London, Greater London, United Kingdom SENIOR SPIRIT OF ROSELLE PARK Full time

    About Senior Spirit of Roselle ParkSenior Spirit of Roselle Park is a leading provider of data science solutions, with a team of over 350 data scientists and 1,000 projects since 2006. Our global presence in Paris, London, NY, and HK enables us to lead projects in over 50 countries across various industries, including automotive, financial services, retail,...