AWS DataOps Specialist

4 weeks ago

London, United Kingdom Norton Blake Full time

AWS DataOps Specialist - £650pd (inside IR35) - Hybrid/London

My client are seeking an experienced AWS DataOps Specialist with a strong background in cloud DevOps, big data administration, and data engineering on AWS. This role is crucial for managing and optimizing big data environments, with a particular focus on AWS Lake Formation, data product concepts, and Spark job management. You will play a key role in creating and maintaining services and tools for a multi-tenant environment, ensuring high performance and scalability.

Key Responsibilities:

Cloud DevOps & Big Data Administration:

  • Manage and optimize big data environments on AWS, ensuring efficient administration and maintenance.
  • Design and implement data lakehouse versus data fabric architectures using AWS Lake Formation, ensuring data integrity and accessibility.

Data Engineering & Spark Management:

  • Develop and maintain Spark jobs, focusing on auditing, monitoring, and performance tuning.
  • Utilize Spark 3+ features, including Adaptive Query Execution (AQE), for job-level resource management and performance optimization.
  • Create services and tools to manage multi-tenant environments, ensuring seamless operations across tenants.

Required Skills:

Cloud DevOps & Big Data:

  • Extensive experience in cloud DevOps and big data administration on AWS.
  • Proficiency in AWS Lake Formation and a strong understanding of data lakehouse versus data fabric concepts.

Programming & Data Engineering:

  • Expertise in Python for data processing and automation.
  • Deep knowledge of Spark internals, including Spark 3+ features like AQE and job-level resource management.

If you are an AWS DataOps Specialist with DevOps experience looking for a new opportunity, please get in touch as soon as possible.

AWS Data Engineer - Contract - London/Hybrid - £650pd (inside IR35)

  • Dataops Engineer

    3 weeks ago

    London, United Kingdom L&G Recruitment Full time

    Job Description :We are looking for a DataOps Engineer to support regulatory data deliverables across several Regulatory Enhancement initiatives (e.g. WORM, Trade Venue Governance, Trade Surveillance, etc).Essential Proficiency in writing database queries using SQL and general understanding of relational databases, and in data pipeline design and...

  • Dataops Engineer

    3 weeks ago

    London, United Kingdom L&G Recruitment Full time

    Job Description :We are looking for a DataOps Engineer to support regulatory data deliverables across several Regulatory Enhancement initiatives (e.g. WORM, Trade Venue Governance, Trade Surveillance, etc).Essential Proficiency in writing database queries using SQL and general understanding of relational databases, and in data pipeline design and...

  • Dataops Engineer

    1 month ago

    London, United Kingdom L&G Recruitment Full time

    Job Description : We are looking for a DataOps Engineer to support regulatory data deliverables across several Regulatory Enhancement initiatives (e.g. WORM, Trade Venue Governance, Trade Surveillance, etc). Essential Proficiency in writing database queries using SQL and general understanding of relational databases, and in data pipeline design...

  • Dataops Engineer

    4 weeks ago

    London,, UK, United Kingdom L&G Recruitment Full time

    Job Description :We are looking for a DataOps Engineer to support regulatory data deliverables across several Regulatory Enhancement initiatives (e.g. WORM, Trade Venue Governance, Trade Surveillance, etc).Essential Proficiency in writing database queries using SQL and general understanding of relational databases, and in data pipeline design and...

  • Dataops Engineer

    1 month ago

    London, United Kingdom L&G Recruitment Full time

    Job Description : We are looking for a DataOps Engineer to support regulatory data deliverables across several Regulatory Enhancement initiatives (e.g. WORM, Trade Venue Governance, Trade Surveillance, etc). Essential Proficiency in writing database queries using SQL and general understanding of relational databases, and in data pipeline design...

  • Dataops Engineer

    4 weeks ago

    London, United Kingdom L&G Recruitment Full time

    Job Description :We are looking for a DataOps Engineer to support regulatory data deliverables across several Regulatory Enhancement initiatives (e.g. WORM, Trade Venue Governance, Trade Surveillance, etc).Essential Proficiency in writing database queries using SQL and general understanding of relational databases, and in data pipeline design and...

  • Dataops Engineer

    1 month ago

    London, United Kingdom L&G Recruitment Full time

    Job Description :We are looking for a DataOps Engineer to support regulatory data deliverables across several Regulatory Enhancement initiatives (e.g. WORM, Trade Venue Governance, Trade Surveillance, etc).Essential Proficiency in writing database queries using SQL and general understanding of relational databases, and in data pipeline design and...

  • Dataops Engineer

    1 month ago

    London Area, United Kingdom L&G Recruitment Full time

    Job Description : We are looking for a DataOps Engineer to support regulatory data deliverables across several Regulatory Enhancement initiatives (e.g. WORM, Trade Venue Governance, Trade Surveillance, etc). Essential Proficiency in writing database queries using SQL and general understanding of relational databases, and in data pipeline design and...

  • Dataops Engineer

    1 month ago

    London Area, United Kingdom L&G Recruitment Full time

    Job Description :We are looking for a DataOps Engineer to support regulatory data deliverables across several Regulatory Enhancement initiatives (e.g. WORM, Trade Venue Governance, Trade Surveillance, etc).Essential Proficiency in writing database queries using SQL and general understanding of relational databases, and in data pipeline design and...

  • Dataops Engineer

    1 month ago

    London Area, United Kingdom L&G Recruitment Full time

    Job Description :We are looking for a DataOps Engineer to support regulatory data deliverables across several Regulatory Enhancement initiatives (e.g. WORM, Trade Venue Governance, Trade Surveillance, etc).Essential Proficiency in writing database queries using SQL and general understanding of relational databases, and in data pipeline design and...

  • Dataops Engineer

    3 weeks ago

    London, United Kingdom L&G Recruitment Full time

    Job Description :Remember to check your CV before applying Also, ensure you read through all the requirements related to this role.We are looking for a DataOps Engineer to support regulatory data deliverables across several Regulatory Enhancement initiatives (e.g. WORM, Trade Venue Governance, Trade Surveillance, etc).Essential Proficiency in writing...

  • Dataops Engineer

    2 weeks ago

    London, United Kingdom L&G Recruitment Full time

    Job Description :Remember to check your CV before applying Also, ensure you read through all the requirements related to this role.We are looking for a DataOps Engineer to support regulatory data deliverables across several Regulatory Enhancement initiatives (e.g. WORM, Trade Venue Governance, Trade Surveillance, etc).Essential Proficiency in writing...

  • Senior DataOps Specialist

    3 weeks ago

    London, Greater London, United Kingdom Investigo Full time

    Job Summary:We are seeking a highly skilled DataOps Engineer to join our team at Investigo. As a DataOps Engineer, you will play a key role in supporting our client's move from legacy systems to a Databricks-based data platform.About the Role:This is a greenfield project, and you will be joining from the ground up. You will need to have strong experience...

  • Senior DataOps Specialist

    3 weeks ago

    London, Greater London, United Kingdom Investigo Full time

    Job Summary:We are seeking a highly skilled DataOps Engineer to join our team at Investigo. As a DataOps Engineer, you will play a key role in supporting our client's move from legacy systems to a Databricks-based data platform.About the Role:This is a greenfield project, and you will be joining from the ground up. You will need to have strong experience...

  • AWS DevOps Engineer

    5 hours ago

    London, Greater London, United Kingdom GL Global Full time

    Junior DevOps EngineerWe are seeking a talented Junior DevOps Engineer to join our team at GL Global. As a key member of our data team, you will collaborate closely with data engineers and AI/ML specialists to ensure the stability, scalability, and security of our infrastructure.Key Responsibilities:Collaborate with the data team to support the development...

  • AWS Data Engineer

    2 weeks ago

    London, Greater London, United Kingdom Onyx-Conseil Full time

    Job Title: AWS Data EngineerJob Summary:We are seeking an experienced AWS Data Engineer to join our team at Onyx-Conseil. As a DataOps Specialist, you will play a crucial role in ensuring the availability, quality, and reliability of data throughout its lifecycle.Key Responsibilities:Collaborate with development teams and operational teams to design,...

  • AWS Data Engineer

    2 weeks ago

    London, Greater London, United Kingdom Onyx-Conseil Full time

    Job Title: AWS Data EngineerJob Summary:We are seeking an experienced AWS Data Engineer to join our team at Onyx-Conseil. As a DataOps Specialist, you will play a crucial role in ensuring the availability, quality, and reliability of data throughout its lifecycle.Key Responsibilities:Collaborate with development teams and operational teams to design,...

  • Junior DevOps

    1 week ago

    London, United Kingdom GL Global Full time

    Junior DevOps / DataOps Engineer (Kubernetes, Terraform) - Get in to Data We are working with a tech scale-up who is looking for a talented Junior DevOps Engineer with a passion for Data. In this role, you will collaborate closely with data engineers and AI/ML specialists to ensure the stability, scalability, and security of our infrastructure. Experience...

  • Junior DevOps

    1 week ago

    London, United Kingdom GL Global Full time

    Junior DevOps / DataOps Engineer (Kubernetes, Terraform) - Get in to Data We are working with a tech scale-up who is looking for a talented Junior DevOps Engineer with a passion for Data. In this role, you will collaborate closely with data engineers and AI/ML specialists to ensure the stability, scalability, and security of our infrastructure. Experience...

  • AWS DevOps Engineer

    1 week ago

    London, Greater London, United Kingdom GL Global Full time

    GL Global is seeking a talented Junior DevOps Engineer to collaborate with our data team. As a key member of our infrastructure team, you will work closely with data engineers and AI/ML specialists to ensure the stability, scalability, and security of our cloud infrastructure. Key responsibilities include: * Collaborating with the data team to support the...