Data Engineer

About the Role

As a Data Engineer - Machine Learning Operations, you will be responsible for overseeing the development and deployment of scalable machine learning solutions. You will work closely with our data science team to ensure that machine learning models are optimized for production use cases. This role requires expertise in both data engineering and machine learning operations.

You will play a key role in ensuring that our machine learning projects are set up for success by automating residual manual steps in the development and production lifecycle. Your tasks will include experiment tracking, model metadata management, and automated artefact deployment using YAML-based CI/CD pipelines and Terraform.

This is an opportunity to work on multiple projects concurrently and use your judgement to determine which projects and teams need most of your time. You will also use your cross-project exposure to feedback to the leadership team to guide understanding, improve consistency, and develop initiatives to improve the community for the future.

Who We're Looking For

We're looking for a highly skilled individual with strong experience delivering and monitoring scalable ML solutions via automated ML Ops. Key skills required include expert knowledge of Python and SQL, including NumPy, Pandas, PySpark, and Spark SQL.

• Expert knowledge of ML Ops frameworks in the following categories:

1. Experiment tracking and model metadata management (MLflow)
2. Orchestration of ML workflows (Metaflow)
3. Data and pipeline versioning (Data Version Control)
4. Model deployment, serving, and monitoring (Kubeflow)

Expert knowledge of automated artefact deployment using YAML-based CI/CD pipelines and TerraformWorking knowledge of one or more ML engineering frameworks: TensorFlow, PyTorch, Keras, Scikit-LearnWorking knowledge of object-oriented programming and unit testing in PythonWorking knowledge of application and information security principles and practices: OWASP for Machine LearningWorking knowledge of Unix-based CLI commands, source control, and scriptingWorking knowledge of containerisation: Docker and container orchestration (Kubernetes)Working knowledge of a cloud data platform: Databricks and a data lakehouse architecture (Delta Lake)Working knowledge of the AWS cloud technology stack:

We offer a competitive salary of £70,000-£90,000 per annum, depending on experience. Other benefits include a discretionary group performance-based bonus, 25 days annual leave (plus Bank Holidays), single cover private medical insurance, and a pension scheme.

---