SC Cleared Databricks Data Engineer – Azure Cloud
1 week ago
Job Title: SC Cleared Databricks Data Engineer – Azure CloudContract Type: 12 month contractDay Rate: Up to £400 a day inside IR35Location: Remote or hybrid (as agreed)Start Date: January 5th 2026Clearance required: Must be holding active SC ClearanceWe are seeking an experienced Databricks Data Engineer to design, build, and optimise large-scale data workflows within the Databricks Data Intelligence Platform.The role focuses on delivering high-performing batch and streaming pipelines using PySpark, Delta Lake, and Azure services, with additional emphasis on governance, lineage tracking, and workflow orchestration. Client information remains confidential.Key ResponsibilitiesBuild and orchestrate Databricks data pipelines using Notebooks, Jobs, and WorkflowsOptimise Spark and Delta Lake workloads through cluster tuning, adaptive execution, scaling, and cachingConduct performance benchmarking and cost optimisation across workloadsImplement data quality, lineage, and governance practices aligned with Unity CatalogDevelop PySpark-based ETL and transformation logic using modular, reusable coding standardsCreate and manage Delta Lake tables with ACID compliance, schema evolution, and time travelIntegrate Databricks assets with Azure Data Lake Storage, Key Vault, and Azure FunctionsCollaborate with cloud architects, data analysts, and engineering teams on end-to-end workflow designSupport automated deployment of Databricks artefacts via CI/CD pipelinesMaintain clear technical documentation covering architecture, performance, and governance configurationRequired Skills and ExperienceStrong experience with the Databricks Data Intelligence PlatformHands-on experience with Databricks Jobs and WorkflowsDeep PySpark expertise, including schema management and optimisationStrong understanding of Delta Lake architecture and incremental design principlesProven Spark performance engineering and cluster tuning capabilitiesUnity Catalog experience (data lineage, access policies, metadata governance)Azure experience across ADLS Gen2, Key Vault, and serverless componentsFamiliarity with CI/CD deployment for DatabricksSolid troubleshooting skills in distributed environmentsPreferred QualificationsExperience working across multiple Databricks workspaces and governed catalogsKnowledge of Synapse, Power BI, or related Azure analytics servicesUnderstanding of cost optimisation for data compute workloadsStrong communication and cross-functional collaboration skills
-
Azure Cloud Engineer
4 weeks ago
nottingham, United Kingdom Digital Waffle Full timeRole: Azure Cloud Engineer Salary: £45,000–£55,000 + Bonus Location: Nottingham (80% Remote) We’re looking for a Cloud Engineer to join a growing Infrastructure function within a tech-driven organisation. This role is ideal for someone who enjoys solving complex cloud challenges, collaborating with a wide range of stakeholders, and helping shape...
-
Azure Cloud Engineer
4 weeks ago
Nottingham, United Kingdom Digital Waffle Full timeRole: Azure Cloud Engineer Salary: £45,000–£55,000 + Bonus Location: Nottingham (80% Remote) We’re looking for a Cloud Engineer to join a growing Infrastructure function within a tech-driven organisation. This role is ideal for someone who enjoys solving complex cloud challenges, collaborating with a wide range of stakeholders, and helping shape...
-
Azure Cloud Engineer
4 weeks ago
Nottingham, United Kingdom Digital Waffle Full timeRole: Azure Cloud EngineerSalary: £45,000–£55,000 + BonusLocation: Nottingham (80% Remote)We’re looking for a Cloud Engineer to join a growing Infrastructure function within a tech-driven organisation. This role is ideal for someone who enjoys solving complex cloud challenges, collaborating with a wide range of stakeholders, and helping shape modern,...
-
Azure Cloud Engineer
1 week ago
Nottingham, United Kingdom Digital Waffle Full timeCloud Engineer (DevOps, Cloud & Infrastructure) Location: Nottingham (80% Remote) We’re looking for a Cloud Engineer to join a growing Infrastructure function within a tech-driven organisation. This role is ideal for someone who enjoys solving complex cloud challenges, collaborating with a wide range of stakeholders, and helping shape modern, scalable...
-
Azure Cloud Engineer
1 week ago
Nottingham, Nottingham, United Kingdom Digital Waffle Full time £45,000 - £55,000 per yearRole: Azure Cloud EngineerSalary: £45,000–£55,000 + BonusLocation: Nottingham (80% Remote)We're looking for a Cloud Engineer to join a growing Infrastructure function within a tech-driven organisation. This role is ideal for someone who enjoys solving complex cloud challenges, collaborating with a wide range of stakeholders, and helping shape modern,...
-
Azure Cloud Senior Engineer
4 weeks ago
Nottingham, United Kingdom Digital Waffle Full timeRole: Azure Cloud Engineer Salary: £45,000–£55,000 + Bonus Nottingham (80% Remote) We’re looking for a Cloud Engineer to join a growing Infrastructure function within a tech-driven organisation. This role is ideal for someone who enjoys solving complex cloud challenges, collaborating with a wide range of stakeholders, and helping shape modern,...
-
Senior Data Platform Engineer
5 days ago
Nottingham, United Kingdom LSEG Full timeSenior Data Platform EngineerQualified professionals are invited to apply for the Senior Data Platform Engineer role at LSEG.About LSEGLSEG (London Stock Exchange Group) is more than a diversified global financial markets infrastructure and data business. We are dedicated, open‑access partners that deliver the services our customers expect. With extensive...
-
Nottingham, United Kingdom Jobs via eFinancialCareers Full timeA leading financial services company in Nottingham seeks a Senior Cloud Data Engineer to design and implement data engineering functions. You will work with semi-structured data, modernizing data products, and lead the software development lifecycle. The ideal candidate has relevant experience in financial services, strong proficiency in Azure technologies,...
-
Senior Data Engineer
1 day ago
Nottingham, United Kingdom RES Full timeGet AI-powered advice on this job and more exclusive features. Description Do you want to work to make Power for Good?We're the world's largest independent renewable energy company. We're driven by a simple yet powerful vision: to create a future where everyone has access to affordable, zero carbon energy.We know that achieving our ambitions would be...
-
Senior Data Engineer
2 days ago
Nottingham, United Kingdom RES Full timeDescriptionDo you want to work to make Power for Good?We're the world's largest independent renewable energy company. We're driven by a simple yet powerful vision: to create a future where everyone has access to affordable, zero carbon energy.We know that achieving our ambitions would be impossible without our people. Because we're tackling some of the...
