Senior Data Engineer

4 weeks ago

Reading, United Kingdom CloudFactory Full time

As a global leader in combining people and technology to help companies realise the disruptive potential of AI, CloudFactory is seeking highly skilled and experienced Senior Data Engineers to join our dynamic team to help mature our data platform.

You will have the opportunity to work with cutting-edge technologies and collaborate with a diverse and talented team to scale our data processing capabilities, whilst creating economic opportunities for talented individuals in developing nations.

Although we provide the option to work remotely, we would prefer this position to be office-based for two days per week.

Responsibilities

The successful candidate will translate user requirements into actionable insights by leveraging existing tools and technologies. Your responsibilities will include: 

  • Regular interaction with PMs/stakeholders to understand their requirements
  • Investigation to ascertain if the requirements can be met using existing technology and data resources
  • Data extraction using Python and other tools like Fivetran
  • Data modelling using SQL and the concept of Star and Snowflake schema
  • Assisting in cost and performance optimization choices within the data warehouse.  Communicating reasoning and logic behind opinions to help make optimal choices
  • Write data quality and other metadata collection systems within data pipelines to promote observability and recoverability of pipelines
  • Write unit and integration tests to bolster system reliability
  • Proactively identify the areas of improvement, propose solutions and manage technical debt
  • Mentoring, guiding and line management of three/four team members.
Tech Stack

Our tech stack:

  • Data extraction and orchestration tools - Fivetran, Prefect
  • Data modelling tool - DBT
  • Data Warehouse - Snowflake
  • Data quality cheeks - dbt, Elementary
  • Data visualisation - Tableau and Quicksight
  • Data catalogue and lineage - Atlan, dbt
  • Source code management - Github
  • Infrastructure as Code - Terraform, Cloudformation
  • Programming Language - Python
  • AWS services (ECS, S3)

Interview Process

Candidates will be expected to complete an initial coding and analytical thinking test, followed by an interview with Data Architect and Senior Product Manager where they will be expected to talk through previous projects, including approaches taken and results achieved. 

Successful candidates will then meet with the Engineering Director before a final interview with the CTO.

The process is expected to take 1-2 weeks.

Requirements

The ideal candidate will possess:

  • Proficiency in requirement capture and technical design
  • Mastery in Data pipeline development
    • Proficiency with extracting data from source systems using Python
    • Mastery in transforming data using SQL and data transformation tools
    • Mastery of data modelling concepts and practices, preferably with Star schema
  • Proficiency in writing data quality tests
  • Proficiency in writing end-to-end tests using test scenarios approach including test data creation and management
  • Proficiency in data pipeline observability techniques (e.g. monitoring, alerting, anomaly detection etc)
  • Some experience in building reports and dashboards
  • Data Warehouse administration
    • Proficiency with Data Warehouse administration (e.g. roles, permissions, compute, performance)
  • Data security and Governance
    • Proficiency in Data security and Governance (e.g. RBAC, encryption, masking, data classification and tagging, time travel and data recovery) practices
    • Some knowledge of using data catalogue and data lineage tools preferably Atlan and dbt documentation
  • Performance and cost optimization
    • Proficiency in performance and cost optimization of database queries using tools and techniques
  • Proficiency with line management to manage a team of Data Engineers and Data Analysts/Analytics Engineers
  • Excellent communication skills, able to translate complex data into understandable insights and narratives.

Benefits

Location dependent, but can include

  • Office snacks provided Monday-Friday, lunches
  • Home work and allowance
  • Medical
  • Amazing Company Mission and Culture
  • Growth Opportunities


Join us, and change the world for the better. If you are skilled and humble, with a commitment to lifelong learning, and you’re curious about the world and its people, you could be a good fit at CloudFactory. We welcome the unique contributions you can bring to help us build a diverse, inclusive workplace because we connect, learn, and grow stronger from our differences. We want you to bring your whole, authentic self to work.

We look forward to hearing from you

Still unsure? Read ‘5 Reasons You Should Work at CloudFactory’.

  • Senior Data Engineer

    1 week ago

    Reading, United Kingdom Austin Fraser Full time

    Senior Data Engineer Bracknell - Flexible  Up to £90,000 Austin Fraser has an exciting opportunity for a Senior Data Engineer to join one of the leading retail chains, running two of the busiests sites in the UK. This is a great time to join the business as they are reshaping the in-house data engineering capability by moving to a cloud native platform,...

  • Data Engineer

    4 weeks ago

    Reading, United Kingdom Xpertise Recruitment Full time

    Xpertise is seeking a series of Data Engineers with cloud experience to join a fledgling team in Reading. As part of our client's growing engineering division, you will play a pivotal role in leading the data engineering capabilities, working closely with Platform Engineers, Developers, and Analysts.Key details:Salary: £(phone number removed) (Mid-Lead)I'd...

  • Senior Data Engineer

    4 weeks ago

    Reading, United Kingdom CloudFactory Full time

    As a global leader in combining people and technology to help companies realise the disruptive potential of AI, CloudFactory is seeking highly skilled and experienced Senior Data Engineers to join our dynamic team to help mature our data platform. You will have the opportunity to work with cutting-edge technologies and collaborate with a diverse and talented...

  • Senior Data Engineer

    2 weeks ago

    Reading, United Kingdom CloudFactory Full time

    As a global leader in combining people and technology to help companies realise the disruptive potential of AI, CloudFactory is seeking highly skilled and experienced Senior Data Engineers to join our dynamic team to help mature our data platform. You will have the opportunity to work with cutting-edge technologies and collaborate with a diverse and talented...

  • Senior Data Engineer

    1 week ago

    Reading, United Kingdom Mirus Talent Full time

    We're representing a dynamic business dedicated to extending healthy lifespans and cutting unnecessary medical costs. They're seeking a Senior Data Engineer to propel their Data team forward.Key Responsibilities:nLead Cloud Data Warehouse Project:Oversee the next development phase, ensuring top-notch scalability, reliability, and performance.nData Management...

  • Senior Data Engineer

    1 week ago

    Reading, United Kingdom Mirus Talent Full time

    We're representing a dynamic business dedicated to extending healthy lifespans and cutting unnecessary medical costs. They're seeking a Senior Data Engineer to propel their Data team forward. Key Responsibilities: Lead Cloud Data Warehouse Project: Oversee the next development phase, ensuring top-notch scalability, reliability, and performance. Data...

  • Senior Data Engineer

    2 weeks ago

    Reading, United Kingdom Mirus Talent Full time

    We're representing a dynamic business dedicated to extending healthy lifespans and cutting unnecessary medical costs. They're seeking a Senior Data Engineer to propel their Data team forward.Key Responsibilities:Lead Cloud Data Warehouse Project: Oversee the next development phase, ensuring top-notch scalability, reliability, and performance.Data Management...

  • Senior Data Engineer

    2 weeks ago

    Reading, United Kingdom Mirus Talent Full time

    We're representing a dynamic business dedicated to extending healthy lifespans and cutting unnecessary medical costs. They're seeking a Senior Data Engineer to propel their Data team forward.Key Responsibilities:Lead Cloud Data Warehouse Project: Oversee the next development phase, ensuring top-notch scalability, reliability, and performance.Data Management...

  • Senior Data Engineer

    2 weeks ago

    Reading, United Kingdom Mirus Talent Full time

    We're representing a dynamic business dedicated to extending healthy lifespans and cutting unnecessary medical costs. They're seeking a Senior Data Engineer to propel their Data team forward.Key Responsibilities:Lead Cloud Data Warehouse Project: Oversee the next development phase, ensuring top-notch scalability, reliability, and performance.Data Management...

  • Senior Data Engineer

    1 week ago

    Reading, United Kingdom Mirus Talent Full time

    We're representing a dynamic business dedicated to extending healthy lifespans and cutting unnecessary medical costs. They're seeking a Senior Data Engineer to propel their Data team forward.The information below covers the role requirements, expected candidate experience, and accompanying qualifications.Key Responsibilities:Lead Cloud Data Warehouse...

  • Senior Data Engineer

    1 week ago

    Reading, United Kingdom Mirus Talent Full time

    We're representing a dynamic business dedicated to extending healthy lifespans and cutting unnecessary medical costs. They're seeking a Senior Data Engineer to propel their Data team forward.The information below covers the role requirements, expected candidate experience, and accompanying qualifications.Key Responsibilities:Lead Cloud Data Warehouse...

  • Senior Data Engineer

    1 week ago

    Reading, United Kingdom Mirus Talent Full time

    We're representing a dynamic business dedicated to extending healthy lifespans and cutting unnecessary medical costs. They're seeking a Senior Data Engineer to propel their Data team forward.Ready to apply Before you do, make sure to read all the details pertaining to this job in the description below.Key Responsibilities:Lead Cloud Data Warehouse Project:...

  • Senior Data Engineer

    1 week ago

    Reading, United Kingdom Mirus Talent Full time

    We're representing a dynamic business dedicated to extending healthy lifespans and cutting unnecessary medical costs. They're seeking a Senior Data Engineer to propel their Data team forward.Ready to apply Before you do, make sure to read all the details pertaining to this job in the description below.Key Responsibilities:Lead Cloud Data Warehouse Project:...

  • Senior Data Engineer

    1 week ago

    Reading, United Kingdom Mirus Talent Full time

    We're representing a dynamic business dedicated to extending healthy lifespans and cutting unnecessary medical costs. They're seeking a Senior Data Engineer to propel their Data team forward.Key Responsibilities:Lead Cloud Data Warehouse Project: Oversee the next development phase, ensuring top-notch scalability, reliability, and performance.Data...

  • Senior Engineer, Data Engineering

    1 week ago

    Reading, Berkshire, United Kingdom Mirus Talent Full time

    Job Description We're representing a dynamic business dedicated to extending healthy lifespans and cutting unnecessary medical costs. They're seeking a Senior Data Engineer to propel their Data team forward. Lead Cloud Data Warehouse Project: Oversee the next development phase, ensuring top-notch scalability, reliability, and performance. Data...

  • Senior Engineer, Data Engineering

    1 week ago

    Reading, Berkshire, United Kingdom Mirus Talent Full time

    We're representing a dynamic business dedicated to extending healthy lifespans and cutting unnecessary medical costs. They're seeking a Senior Data Engineer to propel their Data team forward. Lead Cloud Data Warehouse Project: Oversee the next development phase, ensuring top-notch scalability, reliability, and performance. Data Management Ownership: Be...

  • Senior Engineer, Data Engineering

    1 week ago

    Reading, Berkshire, United Kingdom Mirus Talent Full time

    We're representing a dynamic business dedicated to extending healthy lifespans and cutting unnecessary medical costs. They're seeking a Senior Data Engineer to propel their Data team forward. Lead Cloud Data Warehouse Project: Oversee the next development phase, ensuring top-notch scalability, reliability, and performance. Data Management Ownership: Be...

  • Senior Data Engineer

    1 week ago

    Reading, United Kingdom Thames Water Full time

    **Who are we?** Thames Water is the UK’s largest water and wastewater company. We make a daily difference to our 15 million customers by supplying 2.6 billion litres of water through 32,000 km of pipes_,_ keeping taps flowing and toilets flushing.? At Thames Water, every one of our actions, big and small, matters every day. Water is essential to life, so...

  • Data Engineers

    4 weeks ago

    Reading, Berkshire, United Kingdom Xpertise Recruitment Full time

    Xpertise is seeking a series of Data Engineers with cloud experience to join a fledgling team in Reading. As part of our client's growing engineering division, you will play a pivotal role in leading the data engineering capabilities, working closely with Platform Engineers, Developers, and Analysts. Salary: £(phone number removed) (Mid-Lead) I'd...

  • Data Centre Testing Engineer

    3 weeks ago

    Reading, United Kingdom Ventula Consulting Full time

    Senior Data Centre Testing Engineer – £65-75k + Excellent Benefits - Reading (Hybrid)Our multinational Renewable Energy Client now requires an experienced Senior Data Centre Testing Engineer responsible for developing new tests and installations.The Data Centre Testing Engineer will be responsible for building effective networks within the Technology...