Data Engineer

4 weeks ago

London, United Kingdom Our Future Health Limited Full time €60,000

We are looking for a Data Engineer to join our growing data engineering team at Our Future Health. Our data engineers work with our participant data (including various health records from the NHS, genetic data, Our Future Health’s own questionnaires) and build pipelines and systems to process it, control quality and create data releases for researchers. You’ll support the development of specific pipelines related to questionnaire and physical measurements data.

Our Future Health will be the UK’s largest ever health research programme, bringing people together to develop new ways to detect, prevent, and treat diseases. We are a charity, supported by the UK Government, in partnership with charities and industry. We work closely with the NHS and with public authorities across all nations and regions of the UK.

What you’ll be doing:

We ask Our Future Health volunteers to complete a health questionnaire which is currently delivered through our website and has been completed by hundreds of thousands of people. The questionnaire data is processed using our data platform and made available to researchers working on approved research projects in our trusted research environment. We have some exciting plans to improve the way that the questionnaires are collected as well as the range of questionnaires and will also be looking to improve how the data are presented to researchers.

Your key responsibilities will include but not be limited to:

  • Support the technical product owner for questionnaire and physical measurements data releases, ensuring that requirements around schemas and data processing logic are well specified from a technical perspective.
  • Play a supporting role in developing data pipelines and contributing to the code base.
  • Ensure best practices are used in the team (CI/CD, designing modular and reusable code, etc)
  • Anticipate problems that could occur with data pipelines and take action to prevent them. Identify and describe problems when they occur and be able to develop solutions that address them for the short and long term.
  • Support effective multidisciplinary working with other teams, helping teams understand the role of data engineering in Our Future Health and how to work effectively together.
  • Keep abreast of best practice in data engineering across industry, research and Government, and identify opportunities to bring these into Our Future Health.

You know how to communicate to and between technical and non-technical stakeholders as well as facilitate discussions within a multidisciplinary team including scientists, software engineers, product managers and other data engineers. To succeed in this role, you will also have some of the following skills:

  • Experience working in an agile development team.
  • Understanding and working knowledge of information governance and data security approaches appropriate for sensitive health data.
  • Familiarity with a range of data structures, data standards and data exchange formats, with experience with questionnaire data being desirable (in addition, OMOP and FHIR experience would be advantageous).
  • You have experience with the design and deployment of complex data schemas.
  • You have experience in validating and cleaning complex datasets.
  • Experience building and maintaining robust, scalable and efficient pipelines capable of processing very large amounts of data from one or multiple systems.
  • Highly proficient in Python.
  • Highly proficient in SQL.
  • Experience working with data lakes; experience with Spark, Databricks.
  • Understanding of common data transformation and storage formats, e.g. Apache Parquet.
  • Good understanding of cloud environments (ideally Azure), and workflow management systems (e.g. Dagster, dbt, Airflow, Prefect).
  • Understanding of containerisation (e.g. Docker) and deployment (e.g. Kubernetes).
  • Follow best practices like code review, clean code and unit tests.
  • Experience with version control and Git/GitHub.
  • Up to £60,000 per annum basic salary.
  • Generous company pension package with employer contributions of up to 12%.
  • 30 days annual leave (plus bank holidays.)
  • Individual development budget
  • Flexible and remote working arrangements and a lovely new office in Holborn, Central London.

Join us - let’s prevent disease together.

#J-18808-Ljbffr

  • Data Engineer

    3 weeks ago

    London, United Kingdom Data Idols Full time

    **Senior Data Engineer** Data Idols are proud to be working with an iconic UK brand to assist them in hiring a Senior Data Engineer as they undertake a digital and data-driven journey that will transform the business. Data is at the heart of the organisation and forms a key partof their future plans. **The Opportunity** As a Senior Data Engineer you will be...

  • Data Engineer

    3 weeks ago

    London, United Kingdom Data Idols Full time

    **Data Engineer** Are you a mid-level **Data Engineer** skilled in **Azure technologies**, seeking an new 'feel-good' permanent opportunity? Data Idols are proud to be supporting a HealthTech client, who use remote-first technology focusing on engagingpatients across their healthcare journey while saving hospitals time, effort, and money! This opportunity...

  • Data Engineer

    2 months ago

    London, United Kingdom Data Idols Full time

    **Data Engineer** Data Idols are excited to be working with one of the top 100 companies in the world to recruit a Data Engineer! A company which owns multiple brands and who are investing heavily into 'all things data' to revolutionise their extraordinary team. **Skills and Experience**: - Experience with Azure - Experience with SQL - Experience with...

  • Data Engineer

    4 weeks ago

    London, United Kingdom Data Idols Full time

    **Data Engineer** Data Idols are excited to be working with one of the top 100 companies in the world to recruit a Data Engineer! A company which owns multiple brands and who are investing heavily into 'all things data' to revolutionise their extraordinary team. **Skills and Experience**: - Experience with Azure - Experience with SQL - Experience with...

  • Data Engineer

    1 week ago

    London, Greater London, United Kingdom Data Idols Full time

    Data Engineer Are you a mid-level Data Engineer skilled in Azure technologies, seeking an new 'feel-good' permanent opportunity? Data Idols are proud to be supporting a HealthTech client, who use remote-first technology focusing on engagingpatients across their healthcare journey while saving hospitals time, effort, and money This opportunity provides remote...

  • Data Engineer

    2 months ago

    London, United Kingdom Data Idols Full time

    **Data Engineer** Data Idols are proud to be working with a leading technology and engineering consultancy who have been running for decades globally. They work with a variety of clients across different industries on their biggest challenges. Would you like to work alongsidethis company, where you get the chance to work across a wide range of tools, tech,...

  • Senior Data Engineer

    2 months ago

    London, United Kingdom Data Idols Full time

    **Senior Data Engineer** Data Idols are excited to be working with a well-funded startup who are moving into their scale-up phase. We are working to recruit the first Data Engineer, you will be involved in architecting and building their greenfield data platform. In this excitingand dynamic role, you will be involved from the very beginning, selecting the...

  • Senior Engineer, Data Engineering

    1 week ago

    London, Greater London, United Kingdom Data Idols Full time £85,000

    Job Description Senior Data Engineer Location: London (Hybrid) Data Idols are working with an exciting scale-up revolutionising delivery logistics and moving items across the UK with efficiency and ease all whilst cutting emissions. Trusted by millions of customers already they are expanding their data team to manage increased volumes of data and demand. As...

  • Gcp Data Engineer

    3 weeks ago

    London, United Kingdom Data Idols Full time

    **GCP Data Engineer** Would you like to build enterprise scale data solutions for a global organisation that reaches 400 million users monthly? As a GCP Data Engineer, you will be involved in delivering solutions from initial concept to production. **The Opportunity** Working for a well known brand you will be responsible for all aspects of the data...

  • Lead Engineer/Data Engineer

    3 months ago

    London, United Kingdom Abs Data Full time

    Lead Data Engineer Do you enjoy creating valuable insights with data? Do you want to play a key role in transforming our firm into an agile organization? Being agile will make us more responsive, more adaptable, and ultimately more innovative. We’re looking for a Data Engineer to: ~ transform data into valuable insights that inform business...

  • Senior Engineer, Data Engineering

    3 months ago

    London, United Kingdom Data Idols Full time

    Senior Data Engineer Location: London, 3/2 days in the office per week Data Idols are currently working with a company that focus on the european marketplace for transport who are looking to make the future greener! Within this role as a Senior Data Engineer, you will be monitoring, designing and implementing a datawarehouse, whilst working with...

  • Senior Engineer, Data Engineering

    3 months ago

    London, United Kingdom Data Idols Full time

    Senior Data Engineer Location: London, 3/2 days in the office per week Data Idols are currently working with a company that focus on the european marketplace for transport who are looking to make the future greener! Within this role as a Senior Data Engineer, you will be monitoring, designing and implementing a datawarehouse, whilst working with...

  • Senior Data Engineer

    2 weeks ago

    London, United Kingdom Data Idols Full time €85,000

    Senior Data Engineer Salary: up to £85,000 Location: London (Hybrid) Data Idols are working with an exciting scale-up revolutionising delivery logistics and moving items across the UK with efficiency and ease all whilst cutting emissions. Trusted by millions of customers already they are expanding their data team to manage increased volumes of data and...

  • Senior Data Engineer

    2 weeks ago

    London, United Kingdom Data Idols Full time €85,000

    Senior Data Engineer Salary: up to £85,000 Location: London (Hybrid) Data Idols are working with an exciting scale-up revolutionising delivery logistics and moving items across the UK with efficiency and ease all whilst cutting emissions. Trusted by millions of customers already they are expanding their data team to manage increased volumes of data and...

  • Senior Data Engineer

    2 weeks ago

    London, United Kingdom Data Idols Full time

    Senior Data EngineerSalary: up to £85,000Location: London (Hybrid)Data Idols are working with an exciting scale-up revolutionising delivery logistics and moving items across the UK with efficiency and ease all whilst cutting emissions. Trusted by millions of customers already they are expanding their data team to manage increased volumes of data and demand....

  • Senior Data Engineer

    1 week ago

    London, Greater London, United Kingdom Data Idols Full time

    Senior Data EngineerIncrease your chances of reaching the interview stage by reading the complete job description and applying promptly.Salary: up to £85,000Location: London (Hybrid)Data Idols are working with an exciting scale-up revolutionising delivery logistics and moving items across the UK with efficiency and ease all whilst cutting emissions. Trusted...

  • Senior Data Engineer

    2 months ago

    London, United Kingdom Data Idols Full time

    **Senior Data Engineer** Data Idols are proud to be working with a leading technology and engineering consultancy who have been running for decades globally. They work with a variety of clients across different industries on their biggest challenges. Would you like to work alongsidethis company, where you get the chance to work across a wide range of tools,...

  • Senior Data Engineer

    2 weeks ago

    London, United Kingdom Data Idols Full time

    Senior Data EngineerSalary: up to £85,000Location: London (Hybrid)Data Idols are working with an exciting scale-up revolutionising delivery logistics and moving items across the UK with efficiency and ease all whilst cutting emissions. Trusted by millions of customers already they are expanding their data team to manage increased volumes of data and demand....

  • Senior Data Engineer

    2 weeks ago

    London, United Kingdom Data Idols Full time

    Senior Data EngineerSalary: up to £85,000Location: London (Hybrid)Data Idols are working with an exciting scale-up revolutionising delivery logistics and moving items across the UK with efficiency and ease all whilst cutting emissions. Trusted by millions of customers already they are expanding their data team to manage increased volumes of data and demand....

  • Senior Data Engineer

    2 weeks ago

    London, United Kingdom Data Idols Full time

    Senior Data EngineerLike the look of this opportunity Make sure to apply fast, as a high volume of applications is expected Scroll down to read the complete job description.Salary: up to £85,000Location: London (Hybrid)Data Idols are working with an exciting scale-up revolutionising delivery logistics and moving items across the UK with efficiency and ease...