Senior Data Engineer

2 weeks ago

Ebbw Vale, Blaenau Gwent, United Kingdom Methods Full time
Job Description

Methods Analytics

At Methods Analytics, we're dedicated to improving society by helping people make better decisions with data. Our team combines passionate individuals, sector-specific insight, and technical excellence to provide our customers with an end-to-end data service.

We take a collaborative, creative, and user-centric approach to data, ensuring that our outputs are transparent, robust, and value discussion and debate. We question assumptions, ambition, and process with respect and humility, relishing difficult problems and overcoming them with innovation, creativity, and technical freedom.

We treat data with respect and use it only for the right purpose. Our people are positive, dedicated, and relentless, striving for interactions that are engaging, informative, and fun in equal measure while maintaining a steely focus on outcomes and delivering quality products for our customers.

We're passionate about our people, wanting our colleagues to develop the things they're good at and enjoy.

Key Responsibilities

  • Design, build, and maintain sophisticated data infrastructure systems that operate across both on-premises and Azure cloud environments.
  • Deploy and manage scalable data operations that support advanced analytics and data-driven decision-making.
  • Develop and manage data pipelines using Python, capable of supporting both streaming and batch data processing across structured, semi-structured, and unstructured data in on-premises and Azure environments.
  • Implement and manage data storage solutions leveraging both on-premises infrastructure and Azure, ensuring seamless data integration and accessibility across platforms.
  • Utilise Docker for containerisation and Kubernetes for orchestration, ensuring scalable and efficient deployment of applications across both cloud-based and on-premises environments.
  • Employ tools such as Apache Airflow to automate data flows and manage complex workflows within hybrid environments.
  • Utilise event-driven technologies such as Kafka to handle real-time data streams effectively.
  • Manage security setups and access controls, incorporating tools like Keycloak to protect data integrity and comply with legal standards across all data platforms.
  • Oversee and enhance Elasticsearch setups for robust data searching and analytics capabilities in mixed infrastructure settings.
  • Administer and optimise PostgreSQL databases, ensuring high performance and availability across diverse deployment scenarios.

Requirements

  • Strong Python skills and expertise in scripting and automating data processes across varied environments.
  • Experience with ETL/ELT and developing and optimising ETL or ELT workflows, particularly in hybrid (on-premises and Azure) environments.
  • Expertise in hybrid cloud data architecture and integrating on-premises infrastructure with Azure cloud services.
  • Containerisation and orchestration expertise, including Docker and Kubernetes in managing applications across both on-premises and cloud platforms.
  • Proficiency in workflow automation tools, including Apache Airflow in hybrid data environments.
  • Experience in event streaming and managing and deploying event streaming platforms like Kafka.
  • Data security knowledge and experience with implementing security practices and tools, including Keycloak, across multiple platforms.
  • Search and database management skills, including managing Elasticsearch and PostgreSQL in environments that span on-premises and cloud infrastructures.

Your Impact

In this role, you will empower business leaders to make informed decisions by delivering timely, accurate, and actionable data insights from a robust, hybrid infrastructure. Your expertise will drive the seamless integration of on-premises and cloud-based data solutions, enhancing both the flexibility and scalability of our data operations.

Benefits

  • Autonomy to develop and grow your skills and experience.
  • Be part of exciting project work that is making a difference in society.
  • Strong, inspiring, and thought-provoking leadership.
  • A supportive and collaborative environment.
  • Development opportunities, including access to LinkedIn Learning and a management development programme.
  • Wellness initiatives, including a 24/7 confidential employee assistance programme.
  • Flexible working arrangements, including home working and part-time options.
  • Social activities, including office parties, breakfast Tuesdays, monthly pizza Thursdays, Thirsty Thursdays, and a commitment to charitable causes.
  • Time off, including 25 days of annual leave a year, plus bank holidays, with the option to buy 5 extra days each year.
  • Volunteering opportunities, including 2 paid days per year to volunteer in our local communities or within a charity organisation.
  • Pension scheme, including a Salary Exchange Scheme with 4% employer contribution and 5% employee contribution.
  • Discretionary company bonus, based on company and individual performance.
  • Life assurance, covering 4 times base salary.
  • Private medical insurance, which is non-contributory (spouse and dependants included).
  • Worldwide travel insurance, which is non-contributory (spouse and dependants included).
  • Enhanced maternity and paternity pay.
  • Travel benefits, including a season ticket loan and a cycle to work scheme.

  • Senior Data Engineer

    1 month ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Methods Full time

    About the RoleMethods is seeking a seasoned Senior Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing, building, and maintaining sophisticated data infrastructure systems that operate across both on-premises and Azure cloud environments.Key ResponsibilitiesDevelop and manage data pipelines...

  • Senior Data Engineer

    1 month ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Methods Full time

    About the RoleMethods is seeking a seasoned Senior Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing, building, and maintaining sophisticated data infrastructure systems that operate across both on-premises and Azure cloud environments.Key ResponsibilitiesDevelop and manage data pipelines...

  • Senior Data Architect

    5 days ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Methods Full time

    About the RoleAt Methods, we're on a mission to empower business leaders with data-driven insights. As a Senior Data Engineer, you'll play a pivotal role in designing, building, and maintaining sophisticated data infrastructure systems that operate across on-premises and Azure cloud environments.Key ResponsibilitiesDevelop and manage data pipelines using...

  • Senior Data Engineer

    3 weeks ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Methods Full time

    About the RoleWe are seeking a seasoned Senior Data Engineer to join our team at Methods, a leading provider of data-driven solutions. As a key member of our data engineering team, you will be responsible for designing, building, and maintaining sophisticated data infrastructure systems that operate across both on-premises and Azure cloud environments.Key...

  • Senior Data Engineer

    3 weeks ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Methods Full time

    About the RoleWe are seeking a seasoned Senior Data Engineer to join our team at Methods, a leading provider of data-driven solutions. As a key member of our data engineering team, you will be responsible for designing, building, and maintaining sophisticated data infrastructure systems that operate across both on-premises and Azure cloud environments.Key...

  • Senior Data Engineer

    1 week ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Methods Full time

    About the RoleWe are seeking a seasoned Senior Data Engineer to join our team at Methods. This role is essential for designing, building, and maintaining sophisticated data infrastructure systems that operate across both on-premises and Azure cloud environments.Key ResponsibilitiesDevelop and manage data pipelines using Python, capable of supporting both...

  • Senior Mechanical Engineer

    1 month ago

    Ebbw Vale, Blaenau Gwent, United Kingdom MorePeople Full time

    Job Title: Senior Mechanical EngineerWe are seeking an experienced Senior Mechanical Engineer to join our team on a 4 on 4 off schedule. As a key member of our engineering team, you will be responsible for ensuring efficient production through asset maintenance and technical expertise.Key Responsibilities:Support all site engineering and maintenance...

  • Senior Mechanical Engineer

    1 month ago

    Ebbw Vale, Blaenau Gwent, United Kingdom MorePeople Full time

    Job Title: Senior Mechanical EngineerWe are seeking an experienced Senior Mechanical Engineer to join our team on a 4 on 4 off schedule. As a key member of our engineering team, you will be responsible for ensuring efficient production through asset maintenance and technical expertise.Key Responsibilities:Support all site engineering and maintenance...

  • Senior Software Engineer

    3 days ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Vetro Recruitment Full time

    We are seeking a skilled Senior Software Engineer to join our team at "{company}".Key responsibilities include:Design and develop high-quality software applicationsCollaborate with cross-functional teamsAnalyze and resolve complex technical issuesRequirements:Bachelor's degree in Computer Science or related fieldMinimum 5 years of software development...

  • Senior Mechanical Engineer

    1 month ago

    Ebbw Vale, Blaenau Gwent, United Kingdom MorePeople Full time

    Job Title: Engineer - NightsWe are seeking an experienced Shift Engineer to join our team at one of the UK's leading food manufacturers. This full-time position operates on a 4 on 4 off schedule, offering an excellent opportunity to balance work and personal life.About the Role:As a Shift Engineer, you will support all site engineering and maintenance needs,...

  • Senior Mechanical Engineer

    1 month ago

    Ebbw Vale, Blaenau Gwent, United Kingdom MorePeople Full time

    Job Title: Shift Engineer - NightsWe are seeking an experienced Shift Engineer to join our team at one of the UK's leading food manufacturers. This full-time position operates on a 4 on 4 off schedule, offering an excellent opportunity to balance work and personal life.About the Role:As a Shift Engineer, you will support all site engineering and maintenance...

  • Senior Mechanical Engineer

    1 month ago

    Ebbw Vale, Blaenau Gwent, United Kingdom MorePeople Full time

    Job Title: Engineer - NightsWe are seeking an experienced Shift Engineer to join our team at one of the UK's leading food manufacturers. This full-time position operates on a 4 on 4 off schedule, offering an excellent opportunity to balance work and personal life.About the Role:As a Shift Engineer, you will support all site engineering and maintenance needs,...

  • Senior Mechanical Engineer

    1 month ago

    Ebbw Vale, Blaenau Gwent, United Kingdom MorePeople Full time

    Job Title: Shift Engineer - NightsWe are seeking an experienced Shift Engineer to join our team at one of the UK's leading food manufacturers. This full-time position operates on a 4 on 4 off schedule, offering an excellent opportunity to balance work and personal life.About the Role:As a Shift Engineer, you will support all site engineering and maintenance...

  • Infrastructure Platform Engineer

    1 month ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Methods Full time

    Infrastructure Platform Engineer (On-Prem) ConsultantAt Methods, we're seeking an experienced Infrastructure Platform Engineer to join our team on a permanent basis. As a key member of our team, you'll be responsible for designing, implementing, and managing on-premise infrastructure platforms, applications, and data.Key Responsibilities:Design and implement...

  • Infrastructure Platform Engineer

    1 month ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Methods Full time

    Infrastructure Platform Engineer (On-Prem) ConsultantAt Methods, we're seeking an experienced Infrastructure Platform Engineer to join our team on a permanent basis. As a key member of our team, you'll be responsible for designing, implementing, and managing on-premise infrastructure platforms, applications, and data.Key Responsibilities:Design and implement...

  • Senior Utility Innovations Engineer

    5 days ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Pontoon Full time

    Pontoon is an employment consultancy that champions inclusivity and diversity in the workplace. We help talented individuals shine by showcasing their skills, talents, and unique experiences in a supportive environment that fosters growth and development.Job Overview:We are seeking a highly skilled CORE Engineer to join our client's team on a temporary basis...

  • Infrastructure Platform Engineer

    4 weeks ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Methods Full time

    About the RoleWe are seeking an experienced Infrastructure Platform Engineer to join our team at Methods, a leading IT Services Consultancy. As an Infrastructure Platform Engineer, you will be responsible for designing, implementing, and maintaining on-premise infrastructure solutions for our clients.Key ResponsibilitiesDesign and implement on-premise...

  • Infrastructure Platform Engineer

    4 weeks ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Methods Full time

    About the RoleWe are seeking an experienced Infrastructure Platform Engineer to join our team at Methods, a leading IT Services Consultancy. As an Infrastructure Platform Engineer, you will be responsible for designing, implementing, and maintaining on-premise infrastructure solutions for our clients.Key ResponsibilitiesDesign and implement on-premise...

  • Senior Cost Manager

    3 months ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Sir Robert McAlpine Full time

    Join Our Team as a Senior Cost ManagerIf you are an experienced Senior Cost Manager with a background in overseeing substantial multi-million pound developments, we invite you to explore a remarkable opportunity with Sir Robert McAlpine. This role involves engaging in a significant project that promises to be both challenging and rewarding.Why Choose Us?For...

  • Senior Cost Manager

    2 months ago

    Ebbw Vale, Blaenau Gwent, United Kingdom Sir Robert McAlpine Full time

    Join Our Team as a Senior Cost ManagerIf you are an experienced Senior Cost Manager with a background in overseeing substantial multi-million pound developments, we invite you to explore a remarkable opportunity with Sir Robert McAlpine. This role involves engaging in a significant project that promises to be both challenging and rewarding.Why Choose Us?For...