Current jobs related to Senior Data Engineer AWS,DevOps,Big Data - London, Greater London - Salt Search
-
Senior DevOps Engineer AWS
1 month ago
London, Greater London, United Kingdom Harnham - Data & Analytics Recruitment Full time{"title": "Senior DevOps Engineer AWS", "description": "Job SummaryWe are seeking a highly skilled Senior DevOps Engineer with expertise in AWS to join our team. As a key player in our DevOps/MLOps efforts, you will optimize and manage cloud infrastructure, enabling the efficient deployment of machine learning workflows and data-intensive applications.Key...
-
Senior DevOps Engineer AWS
1 month ago
London, Greater London, United Kingdom Harnham - Data & Analytics Recruitment Full time{"title": "Senior DevOps Engineer AWS", "description": "Job SummaryWe are seeking a highly skilled Senior DevOps Engineer with expertise in AWS to join our team. As a key player in our DevOps/MLOps efforts, you will optimize and manage cloud infrastructure, enabling the efficient deployment of machine learning workflows and data-intensive applications.Key...
-
Data Engineer
4 weeks ago
London, Greater London, United Kingdom Data Privacy Full timeSolutions Engineer (Data Engineering and / or Data Warehousing)About the RoleWe are seeking a highly skilled Solutions Engineer to join our team. As a Solutions Engineer, you will work closely with clients to demonstrate how our Data Intelligence Platform can address complex data challenges.Key ResponsibilitiesDesign and implement data engineering solutions...
-
Data Engineer
4 weeks ago
London, Greater London, United Kingdom Data Privacy Full timeSolutions Engineer (Data Engineering and / or Data Warehousing)About the RoleWe are seeking a highly skilled Solutions Engineer to join our team. As a Solutions Engineer, you will work closely with clients to demonstrate how our Data Intelligence Platform can address complex data challenges.Key ResponsibilitiesDesign and implement data engineering solutions...
-
Data Engineer
1 month ago
London, Greater London, United Kingdom Data Privacy Full timeSolutions Engineer (Data Engineering and / or Data Warehousing)About the RoleWe are seeking a highly skilled Solutions Engineer to join our team. As a Solutions Engineer, you will work closely with clients to demonstrate how our Data Intelligence Platform can address complex data challenges.Key ResponsibilitiesDesign and implement data engineering solutions...
-
Data Engineer
3 weeks ago
London, Greater London, United Kingdom Data Privacy Full timeSolution Architect for Data Engineering and Data WarehousingAt Databricks, we're on a mission to simplify data science and analytics for our customers. As a Solution Architect, you'll work closely with clients to demonstrate how our Data Intelligence Platform can address complex data challenges.Key ResponsibilitiesDesign and implement data engineering...
-
Senior AWS Data Engineer
3 weeks ago
London, Greater London, United Kingdom first moment Fotografie GmbH Full timeJob Title: Senior AWS Data EngineerJob Summary:We are seeking a highly skilled Senior AWS Data Engineer to join our team. As a Senior AWS Data Engineer, you will be responsible for designing, implementing, and maintaining large-scale data engineering solutions on AWS. You will work closely with our data science team to develop and deploy data pipelines, data...
-
AWS Data Engineer
2 months ago
London, Greater London, United Kingdom Capgemini Full timeAbout the RoleWe are seeking a highly skilled AWS Data Engineer to join our Cloud Data Platforms team. As a key member of our team, you will be responsible for designing and developing data pipelines, implementing ETL/ELT processes, and adopting DevOps practices to streamline our data engineering processes.Key ResponsibilitiesDesign and develop data...
-
AWS Data Engineer
2 months ago
London, Greater London, United Kingdom Capgemini Full timeAbout the RoleWe are seeking a highly skilled AWS Data Engineer to join our Cloud Data Platforms team. As a key member of our team, you will be responsible for designing and developing data pipelines, implementing ETL/ELT processes, and adopting DevOps practices to streamline our data engineering processes.Key ResponsibilitiesDesign and develop data...
-
Data Engineer
4 weeks ago
London, Greater London, United Kingdom Data Privacy Full timeUnlock Your Potential as a Data EngineerAt Houseful, we're on a mission to empower individuals to make informed decisions about their homes. As a leading property brand, we're committed to leveraging data and technology to drive the property market forward.About the RoleWe're seeking an experienced Data Engineer to join our team. As a key member of our data...
-
Data Engineer
4 weeks ago
London, Greater London, United Kingdom Data Privacy Full timeUnlock Your Potential as a Data EngineerAt Houseful, we're on a mission to empower individuals to make informed decisions about their homes. As a leading property brand, we're committed to leveraging data and technology to drive the property market forward.About the RoleWe're seeking an experienced Data Engineer to join our team. As a key member of our data...
-
Senior Data Engineer
1 month ago
London, Greater London, United Kingdom Harnham - Data & Analytics Recruitment Full timeSenior Data EngineerAbout the RoleWe are seeking a highly skilled Senior Data Engineer to join our expanding data team. As a Senior Data Engineer, you will be responsible for building and maintaining our AWS platform, working with stakeholders to understand data collected, and implementing best coding practices using CI/CD pipelines.Key...
-
Senior Data Engineer
1 month ago
London, Greater London, United Kingdom Harnham - Data & Analytics Recruitment Full timeSenior Data EngineerAbout the RoleWe are seeking a highly skilled Senior Data Engineer to join our expanding data team. As a Senior Data Engineer, you will be responsible for building and maintaining our AWS platform, working with stakeholders to understand data collected, and implementing best coding practices using CI/CD pipelines.Key...
-
Data Systems Engineer
3 days ago
London, Greater London, United Kingdom Data Privacy Full timeAbout the Role:We are seeking a skilled Data Systems Engineer to join our team at Houseful.Key Responsibilities:Design and implement data warehousing solutions using AWS technologies.Collaborate with senior engineers to ensure data accuracy and implement validation checks.Develop and maintain ETL/ELT pipelines, ensuring alignment with business...
-
Data Engineer
3 weeks ago
London, Greater London, United Kingdom Data Privacy Full timeHouseful is a leading property technology company that aims to empower individuals with data-driven insights. As a Data Engineer, you will play a crucial role in building and maintaining our data infrastructure, ensuring seamless data flow and accuracy.Key Responsibilities:Design and implement ETL/ELT pipelines using AWS technologies to extract, transform,...
-
Senior Data Engineer
4 weeks ago
London, Greater London, United Kingdom Harnham - Data & Analytics Recruitment Full timeSenior Data EngineerAbout the RoleWe are seeking a highly skilled Senior Data Engineer to join our expanding data team. As a Senior Data Engineer, you will be responsible for building and maintaining our AWS platform, working with stakeholders to understand data collected, and implementing best coding practices using CI/CD pipelines.Key...
-
Senior Data Engineer
4 weeks ago
London, Greater London, United Kingdom Harnham - Data & Analytics Recruitment Full timeSenior Data EngineerAbout the RoleWe are seeking a highly skilled Senior Data Engineer to join our expanding data team. As a Senior Data Engineer, you will be responsible for building and maintaining our AWS platform, working with stakeholders to understand data collected, and implementing best coding practices using CI/CD pipelines.Key...
-
AWS Cloud Data Engineer
3 weeks ago
London, Greater London, United Kingdom Capgemini Full timeAbout the RoleWe are seeking a skilled AWS Data Engineer to join our Cloud Data Platforms team. As a key member of our team, you will be responsible for designing and developing data pipelines, implementing ETL/ELT processes, and adopting DevOps practices to streamline our data engineering processes.Key ResponsibilitiesDesign and develop data pipelines to...
-
AWS Cloud Data Engineer
3 weeks ago
London, Greater London, United Kingdom Capgemini Full timeAbout the RoleWe are seeking a highly skilled AWS Data Engineer to join our Cloud Data Platforms team. As a key member of our team, you will be responsible for designing and developing data pipelines, implementing ETL/ELT processes, and adopting DevOps practices to streamline and enhance our data engineering processes.Your ResponsibilitiesDesign and develop...
-
AWS Cloud Data Engineer
3 weeks ago
London, Greater London, United Kingdom Capgemini Full timeAbout the RoleWe are seeking a skilled AWS Data Engineer to join our Cloud Data Platforms team. As a key member of our team, you will be responsible for designing and developing data pipelines, implementing ETL/ELT processes, and adopting DevOps practices to streamline our data engineering processes.Key ResponsibilitiesDesign and develop data pipelines to...
Senior Data Engineer AWS,DevOps,Big Data
2 months ago
We are seeking a highly skilled Senior Data Engineer with expertise in cloud DevOps, big data administration, and data engineering on AWS. The ideal candidate will have a deep understanding of AWS Lake Formation, data product concepts, and Spark job management, including auditing, monitoring, and performance tuning.
Key Responsibilities:- Cloud DevOps & Big Data Administration:
- Manage and optimize big data environments on AWS, with a focus on efficient administration and maintenance.
- Leverage AWS Lake Formation to design and implement data lakehouses versus data fabric architectures, ensuring data integrity and accessibility.
- Data Engineering & Spark Management:
- Develop and maintain Spark jobs, with a focus on auditing, monitoring, and instrumentation to ensure reliability and performance.
- Perform Spark performance tuning, including understanding and applying Spark 3+ features, such as Adaptive Query Execution (AQE) and job-level resource management.
- Create services and tools to manage a multi-tenant environment, ensuring seamless data operations across tenants.
- Infrastructure as Code (IaC):
- Utilize Terraform for infrastructure provisioning and management, ensuring scalable and secure environments.
- Integrate with AWS Glue and HIVE for data processing and management, optimizing workflows for large-scale data operations.
- Data Storage & Management:
- Work with data storage formats like Parquet and Hudi to optimize data storage and retrieval.
- Implement and manage IAM policies for secure data access and management across AWS services.
- Collaboration & Continuous Improvement:
- Collaborate with cross-functional teams, including data scientists, analysts, and other engineers, to develop and deploy data solutions.
- Continuously improve data engineering practices, leveraging new tools and techniques to enhance performance and efficiency.
- Cloud DevOps & Big Data:
- Extensive experience in cloud DevOps and big data administration on AWS.
- Proficiency in AWS Lake Formation, with a strong understanding of data lakehouse versus data fabric concepts.
- Programming & Data Engineering:
- Expertise in Python for data processing and automation.
- Deep knowledge of Spark internals (Spark 3+ features), including architecture, events, system metrics, AQE, and job-level resource management.
- Tech Stack:
- Strong hands-on experience with Terraform for infrastructure management.
- Experience with data formats such as Hudi and Parquet.
- Familiarity with AWS Glue, HIVE, and IAM for data management and security.
- Additional Tools & Technologies:
- Knowledge of Iceberg and its application in data storage and management.
- Experience with Airflow for workflow automation.
- Familiarity with Terragrunt for managing Terraform configurations.
- Understanding of DynamoDB and its integration within data environments.
