Cloud Data Engineer

3 weeks ago

Ebbw Vale, Blaenau Gwent, United Kingdom Methods Full time

About Methods

Methods is a leading provider of data-driven solutions, dedicated to empowering businesses to make informed decisions. Our team of experts combines technical excellence with sector-specific insight to deliver end-to-end data services.

We take a collaborative, creative, and user-centric approach to data, ensuring that our outputs are transparent, robust, and value discussion and debate. We question assumptions, ambition, and process, but do so with respect and humility.

Our Approach

We relish difficult problems and overcome them with innovation, creativity, and technical freedom. Ethics, privacy, and quality are at the heart of our work, and we will not sacrifice these for outcomes. We treat data with respect and use it only for the right purpose.

Our Culture

We are passionate about our people and want our colleagues to develop the things they are good at and enjoy. We strive for interactions that are engaging, informative, and fun in equal measure, while maintaining a steely focus on outcomes and delivering quality products for our customers.

Job Summary

We are seeking a seasoned Senior Data Engineer (Infrastructure) to join our team. This role is essential for designing, building, and maintaining sophisticated data infrastructure systems that operate across both on-premises and Azure cloud environments.

Key Responsibilities

  • Develop and manage data pipelines using Python, capable of supporting both streaming and batch data processing across structured, semi-structured, and unstructured data in on-premises and Azure environments.
  • Implement and manage data storage solutions leveraging both on-premises infrastructure and Azure, ensuring seamless data integration and accessibility across platforms.
  • Utilise Docker for containerisation and Kubernetes for orchestration, ensuring scalable and efficient deployment of applications across both cloud-based and on-premises environments.
  • Employ tools such as Apache Airflow to automate data flows and manage complex workflows within hybrid environments.
  • Utilise event-driven technologies such as Kafka to handle real-time data streams effectively.
  • Manage security setups and access controls, incorporating tools like Keycloak to protect data integrity and comply with legal standards across all data platforms.
  • Oversee and enhance Elasticsearch setups for robust data searching and analytics capabilities in mixed infrastructure settings.
  • Administer and optimise PostgreSQL databases, ensuring high performance and availability across diverse deployment scenarios.

Requirements

  • Strong Python skills and expertise in scripting and automating data processes across varied environments.
  • Experience with ETL/ELT and developing and optimising ETL or ELT workflows, particularly in hybrid (on-premises and Azure) environments.
  • Expertise in hybrid cloud data architecture and integrating on-premises infrastructure with Azure cloud services.
  • Containerisation and orchestration expertise and solid experience with Docker and Kubernetes in managing applications across both on-premises and cloud platforms.
  • Proficiency in workflow automation tools and practical experience with Apache Airflow in hybrid data environments.
  • Experience in event streaming and proven ability in managing and deploying event streaming platforms like Kafka.
  • Data security knowledge and experience with implementing security practices and tools, including Keycloak, across multiple platforms.
  • Search and database management skills and strong background in managing Elasticsearch and PostgreSQL in environments that span on-premises and cloud infrastructures.

Desirable Skills and Experience

  • Certifications in Azure and other relevant technologies.
  • Experience in data engineering and a minimum of 5 years of experience in managing infrastructure in both on-premises and cloud settings.

Security Clearance

This role will require you to have or be willing to go through Security Clearance. As part of the onboarding process, candidates will be asked to complete a Baseline Personnel Security Standard. Details of the evidence required to apply may be found on the government website.

Location

The position requires an on-site presence 5 days per week at our location. If you are passionate about using data to drive decisions and technological innovation in a hybrid infrastructure environment, we encourage you to apply.

  • Senior Data Engineer

    5 days ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Methods Full time

    About the RoleWe are seeking a seasoned Senior Data Engineer (Infrastructure) to join our team at Methods. This role is crucial for designing, building, and maintaining sophisticated data infrastructure systems that operate across both on-premises and Azure cloud environments.Key ResponsibilitiesDesign, construct, and maintain efficient and reliable data...

  • Senior Data Engineer

    5 days ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Methods Full time

    About the RoleWe are seeking a seasoned Senior Data Engineer (Infrastructure) to join our team at Methods. This role is crucial for designing, building, and maintaining sophisticated data infrastructure systems that operate across both on-premises and Azure cloud environments.Key ResponsibilitiesDesign, construct, and maintain efficient and reliable data...

  • Data Infrastructure Specialist

    2 weeks ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Methods Full time

    About MethodsMethods is a leading provider of data-driven solutions, dedicated to empowering businesses to make informed decisions. Our team of experts combines technical excellence with sector-specific insight to deliver end-to-end data services.We take a collaborative, creative, and user-centric approach to data, ensuring that our outputs are transparent,...

  • Data Infrastructure Specialist

    3 weeks ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Methods Full time

    About MethodsMethods is a leading provider of data-driven solutions, dedicated to empowering businesses to make informed decisions. Our team of experts combines technical excellence with sector-specific insight to deliver end-to-end data services.We take a collaborative, creative, and user-centric approach to data, ensuring that our outputs are transparent,...

  • Senior Data Architect

    3 weeks ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Methods Full time

    About MethodsMethods is a leading provider of data-driven solutions, dedicated to empowering businesses to make informed decisions. Our team of experts combines technical excellence with sector-specific insight to deliver end-to-end data services.We take a collaborative, creative, and user-centric approach to data, ensuring that our outputs are transparent,...

  • Senior Data Architect

    2 weeks ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Methods Full time

    About MethodsMethods is a leading provider of data-driven solutions, dedicated to empowering businesses to make informed decisions. Our team of experts combines technical excellence with sector-specific insight to deliver end-to-end data services.We take a collaborative, creative, and user-centric approach to data, ensuring that our outputs are transparent,...

  • Senior Engineer in Packaging Solutions

    1 month ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Energizer Holdings Full time

    Exciting Opportunity for a Senior Engineer in Packaging Solutions at Energizer Holdings, Inc.Are you driven by the challenge of developing groundbreaking packaging strategies? Energizer Holdings is seeking a skilled Senior Engineer in Packaging Solutions to become part of our Global Auto Care division. In this role, you will work alongside a talented team of...

  • Manufacturing Engineer

    1 month ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Energizer Holdings Full time

    What You'll Appreciate About This Role At Energizer Holdings, we take pride in our work and in each other. Our culture fosters continuous learning and improvement to better serve our customers and consumers globally. We are a diverse team of talented individuals from various backgrounds and experiences, and we believe that this diversity is key to our...

  • Manufacturing Engineer

    1 month ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Energizer Holdings Full time

    What You Will Appreciate About This Role At Energizer Holdings, we take pride in our work and foster a culture of collaboration and continuous improvement to meet the needs of our global customers and consumers. Our diverse team comprises talented individuals from various backgrounds and experiences, which enhances our ability to deliver outstanding...

  • Manufacturing Engineer

    1 month ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Energizer Holdings Full time

    What You'll Appreciate About This Role At Energizer Holdings, we foster a culture of passion and collaboration. Our commitment to each other and our continuous pursuit of improvement enable us to meet the diverse needs of our customers globally. Our team is composed of talented individuals from various backgrounds and experiences, and we value this...

  • Nickel Zinc Battery Development Specialist

    3 weeks ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Acorn by Synergie Full time

    About the RoleWe are seeking a highly motivated and detail-oriented Chemical Research Scientist to join our team at Acorn by Synergie. As a Physical Chemist, you will play a vital part in planning and conducting research experiments, assisting in quality control and health and safety, particularly regarding chemicals.Key ResponsibilitiesScale-up and...

  • Physical Chemistry Research Associate

    3 weeks ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Acorn by Synergie Full time

    About the RoleWe are seeking a highly motivated and detail-oriented Chemical Research Scientist to join our team at Acorn by Synergie. As a Physical Chemist, you will play a vital part in planning and conducting research experiments, assisting in quality control and health and safety, particularly regarding chemicals.Key ResponsibilitiesScale-up and...

  • Infrastructure Platform Specialist

    2 days ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Methods Full time

    About the RoleMethods is seeking an experienced Infrastructure Platform Engineer to join our team on a permanent basis. This role will be based on-site and will involve working with our clients to design, implement, and maintain secure and efficient on-premise infrastructure solutions.Key ResponsibilitiesDesign and implement on-premise infrastructure...

  • Infrastructure Platform Specialist

    1 day ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Methods Full time

    About the RoleMethods is seeking an experienced Infrastructure Platform Engineer to join our team on a permanent basis. This role will be based on-site and will involve working with our clients to design, implement, and maintain secure and efficient on-premise infrastructure solutions.Key ResponsibilitiesDesign and implement on-premise infrastructure...

  • Chemical Research Scientist

    3 weeks ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Acorn by Synergie Full time

    Job DescriptionJob Title: Physical ChemistJob Type: Full-timeIndustry: ManufacturingCompany: Acorn by SynergieLocation: Ebbw ValeJob Reference: Not ApplicableJob SummaryWe are seeking a highly motivated and experienced Physical Chemist to join our team in Ebbw Vale. As a Physical Chemist, you will play a vital role in planning and conducting research...

  • Chemical Research Scientist

    3 weeks ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Acorn by Synergie Full time

    Job DescriptionJob Title: Chemical Research ScientistJob Summary:We are seeking a highly motivated and detail-oriented Chemical Research Scientist to join our team at Acorn by Synergie. As a Chemical Research Scientist, you will play a vital role in planning and conducting research experiments, assisting in quality control and health and safety, and...

  • Safety, Health

    1 week ago

    Ebbw Vale, Blaenau Gwent, United Kingdom The Compleat Food Group Full time

    {"title": "Safety, Health & Environment Manager", "content": "About the RoleThe Compleat Food Group is seeking a highly skilled and experienced Safety, Health & Environment Manager to join our team in Ebbw Vale, South Wales. As a key member of our team, you will play a critical role in ensuring compliance with UK legislative requirements and facilitating the...

  • Safety, Health

    1 week ago

    Ebbw Vale, Blaenau Gwent, United Kingdom The Compleat Food Group Full time

    {"title": "Safety, Health & Environment Manager", "content": "About the RoleThe Compleat Food Group is seeking a highly skilled and experienced Safety, Health & Environment Manager to join our team in Ebbw Vale, South Wales. As a key member of our team, you will play a critical role in ensuring compliance with UK legislative requirements and facilitating the...

  • Infrastructure Platform Architect

    3 weeks ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Methods Full time

    About MethodsMethods is a leading IT Services Consultancy with a strong focus on transforming the public sector in the UK. With over 30 years of experience, we pride ourselves on creating innovative and sustainable solutions for our clients. Our team is dedicated to making a positive impact on society through our work.Our MissionWe aim to deliver exceptional...

  • Senior Quality Manager

    3 weeks ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Hunter Selection Full time

    Job DescriptionJob Title: Senior Quality ManagerJob Type: PermanentLocation: Ebbw Vale, Blaenau GwentSalary: £43,000 per annumJob Summary:We are seeking an experienced Senior Quality Manager to join our team at Hunter Selection. As a Senior Quality Manager, you will be responsible for leading the Quality team, liaising with customers, suppliers, and...