AWS Data Engineer

4 weeks ago

London, United Kingdom Salt Full time

Data Engineer (AWS,DevOps,Big Data) – Finance – London Day rate: £550 - £650 inside IR35
Start: ASAP

My new client is looking for a highly skilled Data Engineer with expertise in cloud DevOps, big data administration, and data engineering on AWS. The ideal candidate will have a deep understanding of AWS Lake Formation, data product concepts, and Spark job management, including auditing, monitoring, and performance tuning. This role involves creating and managing services and tools to support a multi-tenant environment, ensuring optimal performance and scalability.

Cloud DevOps & Big Data Administration:
Manage and optimize big data environments on AWS, with a focus on efficient administration and maintenance.
Leverage AWS Lake Formation to design and implement data lakehouses versus data fabric architectures, ensuring data integrity and accessibility.
Data Engineering & Spark Management:
Develop and maintain Spark jobs, with a focus on auditing, monitoring, and instrumentation to ensure reliability and performance.
Perform Spark performance tuning, including understanding and applying Spark 3+ features, such as Adaptive Query Execution (AQE) and job-level resource management.
Create services and tools to manage a multi-tenant environment, ensuring seamless data operations across tenants.
Integrate with AWS Glue and HIVE for data processing and management, optimizing workflows for large-scale data operations.
Data Storage & Management:
Work with data storage formats like Parquet and Hudi to optimize data storage and retrieval.
Implement and manage IAM policies for secure data access and management across AWS services.
Collaborate with cross-functional teams, including data scientists, analysts, and other engineers, to develop and deploy data solutions.
Continuously improve data engineering practices, leveraging new tools and techniques to enhance performance and efficiency.

Cloud DevOps & Big Data:
Extensive experience in cloud DevOps and big data administration on AWS.
Proficiency in AWS Lake Formation, with a strong understanding of data lakehouse versus data fabric concepts.
Programming & Data Engineering:
Expertise in Python for data processing and automation.
Experience with data formats such as Hudi and Parquet.
Familiarity with AWS Glue, HIVE, and IAM for data management and security.

Knowledge of Iceberg and its application in data storage and management.
Familiarity with Terragrunt for managing Terraform configurations.
Understanding of DynamoDB and its integration within data environments.

  • Staff Data Engineer

    2 weeks ago

    London, Greater London, United Kingdom Harnham - Data & Analytics Recruitment Full time

    About the RoleWe are seeking a highly skilled Staff Data Engineer to join our expanding data team at Harnham - Data & Analytics Recruitment. As a Staff Data Engineer, you will be responsible for building and maintaining our company's AWS platform, working closely with stakeholders to understand their data needs, and implementing best coding practices using...

  • Staff Data Engineer

    2 weeks ago

    London, Greater London, United Kingdom Harnham - Data & Analytics Recruitment Full time

    About the RoleWe are seeking a highly skilled Staff Data Engineer to join our expanding data team at Harnham - Data & Analytics Recruitment. As a Staff Data Engineer, you will be responsible for building and maintaining our company's AWS platform, working closely with stakeholders to understand their data needs, and implementing best coding practices using...

  • Staff Data Engineer

    2 weeks ago

    London, Greater London, United Kingdom Harnham - Data & Analytics Recruitment Full time

    About the RoleHarnham - Data & Analytics Recruitment is seeking a highly skilled Staff Data Engineer to join our client's expanding data team. As a Staff Data Engineer, you will be responsible for building and maintaining the company's AWS platform, working closely with stakeholders to understand data collected, and implementing best coding practices using...

  • Staff Data Engineer

    2 weeks ago

    London, Greater London, United Kingdom Harnham - Data & Analytics Recruitment Full time

    About the RoleHarnham - Data & Analytics Recruitment is seeking a highly skilled Staff Data Engineer to join our client's expanding data team. As a Staff Data Engineer, you will be responsible for building and maintaining the company's AWS platform, working closely with stakeholders to understand data collected, and implementing best coding practices using...

  • AWS Data Engineer

    3 days ago

    London, Greater London, United Kingdom Amber Labs Full time

    Job Title: AWS Data EngineerAbout the Role:We are seeking a skilled AWS Data Engineer to join our team at Amber Labs. As a key member of our data engineering team, you will design, implement, and maintain scalable data pipelines and solutions using Amazon Web Services (AWS). Your expertise in data engineering and AWS services will ensure that our data...

  • AWS Data Engineer

    3 days ago

    London, Greater London, United Kingdom Amber Labs Full time

    Job Title: AWS Data EngineerAbout the Role:We are seeking a skilled AWS Data Engineer to join our team at Amber Labs. As a key member of our data engineering team, you will design, implement, and maintain scalable data pipelines and solutions using Amazon Web Services (AWS). Your expertise in data engineering and AWS services will ensure that our data...

  • AWS Data Engineer

    7 days ago

    London, Greater London, United Kingdom Amber Labs Full time

    Job Title: AWS Data EngineerAbout the Role:We are seeking a skilled AWS Data Engineer to join our team at Amber Labs. As a key member of our data engineering team, you will design, implement, and maintain scalable data pipelines and solutions using Amazon Web Services (AWS). Your expertise in data engineering and AWS services will ensure that our data...

  • AWS Data Engineer

    7 days ago

    London, Greater London, United Kingdom Amber Labs Full time

    Job Title: AWS Data EngineerAbout the Role:We are seeking a skilled AWS Data Engineer to join our team at Amber Labs. As a key member of our data engineering team, you will design, implement, and maintain scalable data pipelines and solutions using Amazon Web Services (AWS). Your expertise in data engineering and AWS services will ensure that our data...

  • AWS Data Engineer

    6 days ago

    London, Greater London, United Kingdom Capgemini Full time

    About the RoleWe are seeking a highly skilled AWS Data Engineer to join our Cloud Data Platforms team. As a key member of our team, you will be responsible for designing and developing data pipelines, implementing ETL/ELT processes, and adopting DevOps practices to streamline our data engineering processes.Key ResponsibilitiesDesign and develop data...

  • AWS Data Engineer

    6 days ago

    London, Greater London, United Kingdom Capgemini Full time

    About the RoleWe are seeking a highly skilled AWS Data Engineer to join our Cloud Data Platforms team. As a key member of our team, you will be responsible for designing and developing data pipelines, implementing ETL/ELT processes, and adopting DevOps practices to streamline our data engineering processes.Key ResponsibilitiesDesign and develop data...

  • AWS Data Engineer

    5 days ago

    London, Greater London, United Kingdom Amber Labs Full time

    AWS Data EngineerWe are seeking a skilled AWS Data Engineer to design, implement, and maintain scalable data pipelines and solutions using Amazon Web Services (AWS).The ideal candidate will have a strong background in data engineering and a deep understanding of AWS services, ensuring that data solutions are efficient, secure, and aligned with business...

  • AWS Data Engineer

    4 days ago

    London, Greater London, United Kingdom Amber Labs Full time

    AWS Data EngineerWe are seeking a skilled AWS Data Engineer to design, implement, and maintain scalable data pipelines and solutions using Amazon Web Services (AWS).The ideal candidate will have a strong background in data engineering and a deep understanding of AWS services, ensuring that data solutions are efficient, secure, and aligned with business...

  • AWS Data Engineer

    4 days ago

    London, Greater London, United Kingdom Amber Labs Full time

    AWS Data EngineerWe are seeking a skilled AWS Data Engineer to design, implement, and maintain scalable data pipelines and solutions using Amazon Web Services (AWS).Key Responsibilities:Design and build scalable data pipelines using AWS services like AWS Glue, Lambda, Kinesis, Redshift, and S3.Develop ETL/ELT processes to ingest data from diverse sources...

  • AWS Data Engineer

    4 days ago

    London, Greater London, United Kingdom Amber Labs Full time

    AWS Data EngineerWe are seeking a skilled AWS Data Engineer to design, implement, and maintain scalable data pipelines and solutions using Amazon Web Services (AWS).Key Responsibilities:Design and build scalable data pipelines using AWS services like AWS Glue, Lambda, Kinesis, Redshift, and S3.Develop ETL/ELT processes to ingest data from diverse sources...

  • AWS Data Engineer

    3 days ago

    London, Greater London, United Kingdom Amber Labs Full time

    {"Data Engineer Role": "We are seeking a skilled AWS Data Engineer to design, implement, and maintain scalable data pipelines and solutions using Amazon Web Services (AWS).", "Responsibilities": " * Design, build, and optimize scalable data pipelines using AWS services like AWS Glue, Lambda, Kinesis, Redshift, and S3. * Develop ETL/ELT processes to ingest...

  • AWS Data Engineer

    3 days ago

    London, Greater London, United Kingdom Amber Labs Full time

    {"Data Engineer Role": "We are seeking a skilled AWS Data Engineer to design, implement, and maintain scalable data pipelines and solutions using Amazon Web Services (AWS).", "Responsibilities": " * Design, build, and optimize scalable data pipelines using AWS services like AWS Glue, Lambda, Kinesis, Redshift, and S3. * Develop ETL/ELT processes to ingest...

  • AWS Data Engineer

    18 hours ago

    London, Greater London, United Kingdom Amber Labs Full time

    {"Data Engineer Role": "We are seeking a skilled AWS Data Engineer to design, implement, and maintain scalable data pipelines and solutions using Amazon Web Services (AWS).", "Key Responsibilities": " * Design, build, and optimize scalable data pipelines using AWS services like AWS Glue, Lambda, Kinesis, Redshift, and S3. * Develop ETL/ELT processes to...

  • AWS Data Engineer

    20 hours ago

    London, Greater London, United Kingdom Amber Labs Full time

    {"Data Engineer Role": "We are seeking a skilled AWS Data Engineer to design, implement, and maintain scalable data pipelines and solutions using Amazon Web Services (AWS).", "Key Responsibilities": " * Design, build, and optimize scalable data pipelines using AWS services like AWS Glue, Lambda, Kinesis, Redshift, and S3. * Develop ETL/ELT processes to...

  • AWS Data Engineer

    2 hours ago

    London, Greater London, United Kingdom Capgemini Full time

    About the RoleCapgemini is seeking a highly skilled AWS Data Engineer to join our Cloud Data Platforms team. As a key member of our team, you will be responsible for designing and developing data pipelines, implementing ETL/ELT processes, and adopting DevOps practices to streamline our data engineering processes.Key ResponsibilitiesDesign and develop data...

  • AWS Data Engineer

    3 days ago

    London, Greater London, United Kingdom Amber Labs Full time

    {"Data Engineer Role": "We are seeking a skilled AWS Data Engineer to design, implement, and maintain scalable data pipelines and solutions using Amazon Web Services (AWS).", "Key Responsibilities": " * Design, build, and optimize scalable data pipelines using AWS services like AWS Glue, Lambda, Kinesis, Redshift, and S3. * Develop ETL/ELT processes to...