Senior Data Engineer

4 weeks ago

Edinburgh, United Kingdom Methods Full time

Job Description

Senior Data Engineer (MongoDB / Postgres)

Methods are looking for a Senior Data Engineer to work on an initial 6 month programme of work with a client of ours.

The role will be completed with a hybrid way of working, with the successful candidate needing to be on site in the Edinburgh based office location.

The role has been confirmed to fall outside IR35.

Role Overview

Database Engineer with solid commercial database administration and development experience is required to join the Database Development (DD) team and accelerate our move from legacy on-prem databases to new AWS cloud-native solutions but also maintain and support the existing on-prem DB estate that runs on RHEL. This might be a developer with strong database design and administration skills, or vice-versa.

We operate multiple database technologies – MongoDB, Postgres, Aurora Postgres RDS, Oracle & Informix and are looking for candidates that can work across all of these applications with the team focus on providing DBAAS (Database as a Service) to multiple development teams. Database Development put a strong emphasis on using IaC tools and our on-prem estate is maintained by Ansible, with AWS deployments using AWS CDK. A key focus of the team is developing shared CDK constructs (Typescript) that are consumed by multiple development teams across the organisation for new and migrated databases, so experience with the AWS Developer toolset (CodePipeline, CodeBuild, CodeCommit and CloudFormation) will be useful.

The constructs will embody practical standards for database architecture and quality, allowing centralised assurance. In addition to the listed database technologies – we are currently involved in rolling out a new centralised backup solution using the CommVault application and for monitoring we utilise the TIG stack (Telegraf, InfluxDB & Grafana). The team Grafana instance is deployed using RedHat OpenShift – so experience with containers would be beneficial.

Main Objectives

- Major version upgrade of the current on-prem MongoDB Estate

- Major version upgrades to multiple on-prem Postgres databases

- Support development teams with on-prem migrations to AWS RDS

-Help migrate our current on-prem Grafana instance to containers deployed on OpenShift

- Support the Database team with ongoing BAU tasks such as upgrading, patching, automation, and monitoring improvements etc.

- Help fixing the Support Requests raised by our stakeholders.

Additional Services

- Collaborate with colleagues, to define, refine and prioritise the database roadmap and backlog.

- Actively participate in all team events, leading where specialist knowledge is required and supporting the team to improve their process through inspection and adaptation.

- Engage with the wider communities of practice and interest to share knowledge, technique, and experience.

- Knowledge share with other team members as necessary.

- Developing appropriate documentation and artefacts, e.g., data standards, local work instructions.

- Support updating, installing, and testing new versions and upgrades to databases.

- Troubleshooting and resolving database related issues.

- Ensure existing on-prem estate continues to meet service standards until migration to AWS has been completed.

- Ensure CDK constructs embody agreed standards for resiliency, security, and backups, and meet agreed standards for quality.

- Support and deliver the disaster recover assurance of digital services, striving towards a sustainable Recovery Time Objective of 2hrs and Recovery Point objective of zero.

Essential Experience

Significant commercial experience with the following technology:

- MongoDB

- Postgres

- AWS IAM, S3, EC2, RDS

- Ansible

- Typescript

- CDK and AWS Development tools including Cloud Formation

- SQL

- Monitoring solutions (e.g., CloudWatch, Grafana)

Significant and demonstrable experience in the following areas:

- Making excellent quality software with other people.

- Working with business stakeholders, to understand the context and to focus on delivering value.

- Design and implementation of solutions using a service-based and serverless architecture.

- Excellent collaboration, communication, and teamwork skills

- Experience working in the Agile delivery models - such as Scrum and/or Kanban frameworks.

- Cloud database monitoring, telemetry, intelligence tools and processes

- Defining infrastructure-as-code

- Testing of disaster recovery procedures

- Data replication and high-availability technology

Desirable Skills

Experience delivering new software that consumes services from a centralised database management team.

- Commercial experience of the following:

- Informix

- Oracle

- OpenSearch

- Serverless (Lambda)

#J-18808-Ljbffr

  • Data Scientist

    5 days ago

    Edinburgh, United Kingdom Smart Data Foundry Full time

    About Smart Data FoundryRead the overview of this opportunity to understand what skills, including and relevant soft skills and software package proficiencies, are required.Smart Data Foundry has a purpose to unlock the power of financial data as a force to improve people’s lives.What we doWe enable the research ecosystem to flourish through the provision...

  • Data Scientist

    7 days ago

    Edinburgh, United Kingdom Smart Data Foundry Full time

    About Smart Data Foundry Smart Data Foundry has a purpose to unlock the power of financial data as a force to improve people’s lives. What we do We enable the research ecosystem to flourish through the provision of research ready real financial data. We create data-driven insights based on real financial data, that identify areas to inform policy...

  • Data Scientist

    6 days ago

    Edinburgh, United Kingdom Smart Data Foundry Full time

    About Smart Data Foundry Smart Data Foundry has a purpose to unlock the power of financial data as a force to improve people’s lives. What we do We enable the research ecosystem to flourish through the provision of research ready real financial data. We create data-driven insights based on real financial data, that identify areas to inform policy...

  • Marketing Data Scientist

    6 days ago

    Edinburgh, United Kingdom Smart Data Foundry Full time

    About Smart Data Foundry Smart Data Foundry has a purpose to unlock the power of financial data as a force to improve people’s lives. We enable the research ecosystem to flourish through the provision of research ready real financial data. We create data-driven insights based on real financial data, that identify areas to inform policy change and...

  • Marketing Data Scientist

    7 days ago

    Edinburgh, United Kingdom Smart Data Foundry Full time

    About Smart Data Foundry Smart Data Foundry has a purpose to unlock the power of financial data as a force to improve people’s lives. We enable the research ecosystem to flourish through the provision of research ready real financial data. We create data-driven insights based on real financial data, that identify areas to inform policy change and...

  • Data Scientist

    7 days ago

    Edinburgh, United Kingdom Smart Data Foundry Full time

    About Smart Data FoundrySmart Data Foundry has a purpose to unlock the power of financial data as a force to improve people’s lives.What we doWe enable the research ecosystem to flourish through the provision of research ready real financial data.We create data-driven insights based on real financial data, that identify areas to inform policy change and...

  • Data Scientist

    6 days ago

    Edinburgh, United Kingdom Smart Data Foundry Full time

    Job DescriptionAbout Smart Data FoundrySmart Data Foundry has a purpose to unlock the power of financial data as a force to improve people’s lives.What we doWe enable the research ecosystem to flourish through the provision of research ready real financial data.We create data-driven insights based on real financial data, that identify areas to inform...

  • Data Scientist

    6 days ago

    Edinburgh, United Kingdom Smart Data Foundry Full time

    About Smart Data FoundrySmart Data Foundry has a purpose to unlock the power of financial data as a force to improve people’s lives.What we doWe enable the research ecosystem to flourish through the provision of research ready real financial data.We create data-driven insights based on real financial data, that identify areas to inform policy change and...

  • Data Scientist

    7 days ago

    Edinburgh, United Kingdom Smart Data Foundry Full time

    About Smart Data FoundrySmart Data Foundry has a purpose to unlock the power of financial data as a force to improve people’s lives.What we doWe enable the research ecosystem to flourish through the provision of research ready real financial data.We create data-driven insights based on real financial data, that identify areas to inform policy change and...

  • Data Scientist

    6 days ago

    Edinburgh, United Kingdom Smart Data Foundry Full time

    About Smart Data Foundry Smart Data Foundry has a purpose to unlock the power of financial data as a force to improve people’s lives. What we do We enable the research ecosystem to flourish through the provision of research ready real financial data. We create data-driven insights based on real financial data, that identify areas to inform policy change...

  • Data Scientist

    7 days ago

    Edinburgh, United Kingdom Smart Data Foundry Full time

    About Smart Data FoundrySmart Data Foundry has a purpose to unlock the power of financial data as a force to improve people’s lives.What we doWe enable the research ecosystem to flourish through the provision of research ready real financial data.We create data-driven insights based on real financial data, that identify areas to inform policy change and...

  • Data Platform Engineer

    3 weeks ago

    Edinburgh, United Kingdom Zonal Retail Data Systems Full time

    What you’ll do We are looking for a Data Platform Engineer, someone who can deliver operational excellence and stability in our data platforms that underpin the services we deliver for our customers. You’ll be part of the team helping to build strong, collaborative relationships with architects, operations, and engineering teams. You’ll also...

  • Senior Data Engineer

    4 weeks ago

    Edinburgh, United Kingdom Best Buy Full time

    As a Senior Data Engineer you will be revamping our patient admissions/enrollment capabilities - we’d like to introduce support for net new fields captured within that process, as well as define the most appropriate locations for that data to be stored so that it can be used within different workflows of the future. As a specialist in data engineering,...

  • Data Platform Engineer

    4 weeks ago

    Edinburgh, United Kingdom Zonal Retail Data Systems Limited Full time

    What you’ll do We are looking for a Data Platform Engineer, someone who can deliver operational excellence and stability in our data platforms that underpin the services we deliver for our customers. You’ll be part of the team helping to build strong, collaborative relationships with architects, operations, and engineering teams. You’ll also play a...

  • Senior Data Engineer

    2 weeks ago

    Edinburgh, United Kingdom The Scottish Government Full time

    Overview Have you considered working for Scotland’s devolved tax authority where the work we do and the taxes we collect positively contribute to Scotland’s public services? Do you, like Revenue Scotland, have a passion to make the best use of digital technologies? If so, we would love to hear from you. Based within Revenue Scotland’s DDaT...

  • Senior SQL SSIS Data Engineer

    4 weeks ago

    Edinburgh, United Kingdom Digital Preservation Coalition Full time

    Vacancy for Senior Data Engineer at National Records of Scotland Vacancy for Senior Data Engineer at National Records of Scotland Full-Time National Records of Scotland (NRS) is a Non-Ministerial Department of the Scottish Government, supporting the Registrar General for Scotland and the Keeper of the Records for Scotland. NRS is the Scottish...

  • Data Platform Engineer

    4 weeks ago

    Edinburgh, United Kingdom Zonal Retail Data Systems Limited Full time

    We are looking for a Data Platform Engineer, someone who can deliver operational excellence and stability in our data platforms that underpin the services we deliver for our customers. You’ll be part of the team helping to build strong, collaborative relationships with architects, operations, and engineering teams. As well as the day-to-day you will also...

  • Senior Data Engineer

    2 weeks ago

    Edinburgh, United Kingdom UK Civil Service Full time

    Job summaryHave you considered working for Scotland�s devolved tax authority where the work we do and the taxes we collect positively contribute to Scotland�s public services? Do you, like Revenue Scotland, have a passion to make the best use of digital technologies? If so, we would love to hear from you.Based within Revenue Scotland�s DDaT team, the...

  • Senior Engineering Manager

    4 weeks ago

    Edinburgh, United Kingdom Fairhurst Full time

    Due to a healthy forward workload, we are currently looking to appoint a Principal Bridge Engineer in our Edinburgh office. The successful candidate will have a fantastic opportunity to work in one of the UK’s leading private Engineering Consultancies working on a varied portfolio of bridge works including major infrastructure and highway schemes...

  • Senior Azure Data Engineer

    3 weeks ago

    Edinburgh, United Kingdom Storm ID Full time

    The role As a Data Engineer at Storm you will work alongside other software and cloud infrastructure engineers to design, build and maintain data pipelines, data management solutions and analytics components for our client’s projects across the healthcare, justice and local government sectors. We know that many Data Engineer's will come from a variety...