SC Cleared Databricks Data Engineer – Azure Cloud

5 days ago

England, United Kingdom Montash Full time

Job Title: SC Cleared Databricks Data Engineer – Azure CloudContract Type: 12 month contractDay Rate: Up to £400 a day inside IR35Location: Remote or hybrid (as agreed)Start Date: January 5th 2026Clearance required: Must be holding active SC ClearanceWe are seeking an experienced Databricks Data Engineer to design, build, and optimise large-scale data workflows within the Databricks Data Intelligence Platform.The role focuses on delivering high-performing batch and streaming pipelines using PySpark, Delta Lake, and Azure services, with additional emphasis on governance, lineage tracking, and workflow orchestration. Client information remains confidential.Key ResponsibilitiesBuild and orchestrate Databricks data pipelines using Notebooks, Jobs, and WorkflowsOptimise Spark and Delta Lake workloads through cluster tuning, adaptive execution, scaling, and cachingConduct performance benchmarking and cost optimisation across workloadsImplement data quality, lineage, and governance practices aligned with Unity CatalogDevelop PySpark-based ETL and transformation logic using modular, reusable coding standardsCreate and manage Delta Lake tables with ACID compliance, schema evolution, and time travelIntegrate Databricks assets with Azure Data Lake Storage, Key Vault, and Azure FunctionsCollaborate with cloud architects, data analysts, and engineering teams on end-to-end workflow designSupport automated deployment of Databricks artefacts via CI/CD pipelinesMaintain clear technical documentation covering architecture, performance, and governance configurationRequired Skills and ExperienceStrong experience with the Databricks Data Intelligence PlatformHands-on experience with Databricks Jobs and WorkflowsDeep PySpark expertise, including schema management and optimisationStrong understanding of Delta Lake architecture and incremental design principlesProven Spark performance engineering and cluster tuning capabilitiesUnity Catalog experience (data lineage, access policies, metadata governance)Azure experience across ADLS Gen2, Key Vault, and serverless componentsFamiliarity with CI/CD deployment for DatabricksSolid troubleshooting skills in distributed environmentsPreferred QualificationsExperience working across multiple Databricks workspaces and governed catalogsKnowledge of Synapse, Power BI, or related Azure analytics servicesUnderstanding of cost optimisation for data compute workloadsStrong communication and cross-functional collaboration skills

  • SC Cleared Databricks Data Engineer

    3 days ago

    England, United Kingdom Montash Full time

    Job Title: SC Cleared Databricks Data Engineer – Azure Cloud Contract Type: 12 month contract Day Rate: Up to £400 a day inside IR35 Location: Remote or hybrid (as agreed) Start Date: January 5th 2026 Clearance required: Must be holding active SC Clearance We are seeking an experienced Databricks Data Engineer to design, build, and optimise large-scale...

  • SC Cleared Databricks Data Engineer – Azure Cloud

    5 days ago

    England, United Kingdom Montash Full time

    Job Title: SC Cleared Databricks Data Engineer – Azure Cloud Contract Type: 12 month contract Day Rate: Up to £400 a day inside IR35 Location: Remote or hybrid (as agreed) Start Date: January 5th 2026 Clearance required: Must be holding active SC Clearance We are seeking an experienced Databricks Data Engineer to design, build, and optimise large-scale...

  • Databricks Engineer

    4 weeks ago

    England, United Kingdom iO Associates Full time

    Databricks Engineer - SC Cleared Fully Remote Duration: 6 Months Location: London Rate: £500-£550 per day (Inside IR35) Start Date: ASAP Are you an SC Cleared Databricks Engineer with a passion for building scalable data solutions? Do you have extensive experience in designing and optimising data pipelines, cloud-native platforms, and advanced analytics...

  • Azure Cloud Data Engineer

    3 days ago

    England, United Kingdom Montash Full time

    Job Title: SC Cleared Databricks Data Engineer – Azure Cloud Contract Type: 12 month contract Day Rate: Up to £400 a day inside IR35 Location: Remote or hybrid (as agreed) Start Date: January 5th 2026 Clearance required: Must be holding active SC Clearance We are seeking an experienced Databricks Data Engineer to design, build, and optimise...

  • SC cleared

    2 weeks ago

    England, United Kingdom Methods Full time

    Duration: initial contract until end of March 26, with possible extensionDaily rate : £495 inside IR35Location: RemoteClearance: active SC Key Responsibilities:Design, develop, and optimize data pipelines using Microsoft Fabric and Azure services (eg, Azure Data Factory, Azure Synapse Analytics, Azure Databricks).Build and maintain scalable,...

  • SC cleared

    4 weeks ago

    England, United Kingdom Methods Full time

    Duration: initial contract until end of March 26, with possible extension Daily rate : £495 inside IR35 Location: Remote Clearance : active SC Key Responsibilities: Design, develop, and optimize data pipelines using Microsoft Fabric and Azure services (eg, Azure Data Factory, Azure Synapse Analytics, Azure Databricks). Build and maintain scalable,...

  • Senior Data

    3 days ago

    England, United Kingdom Cognitive Group | Part of the Focus Cloud Group Full time

    Senior AI & Data Consultant - Microsoft Fabric Permanent - Fulltime Homebased - Hybrid working in the UK (No sponsorship can be offered) - Must be SC Cleared or SC Clearable As a Senior Azure AI & Data Consultant you would be designing and delivering next-generation data solutions with Azure Synapse , Azure Databricks , and Microsoft Fabric . This is an...

  • Senior Data

    3 weeks ago

    england, United Kingdom Cognitive Group | Part of the Focus Cloud Group Full time

    Senior AI & Data Consultant - Microsoft Fabric Permanent - Fulltime Homebased - Hybrid working in the UK (No sponsorship can be offered) - Must be SC Cleared or SC Clearable As a Senior Azure AI & Data Consultant you would be designing and delivering next-generation data solutions with Azure Synapse , Azure Databricks , and Microsoft Fabric . This is an...

  • Senior Data

    3 days ago

    England, United Kingdom Cognitive Group | Part of the Focus Cloud Group Full time

    Senior AI & Data Consultant - Microsoft FabricPermanent - FulltimeHomebased - Hybrid working in the UK(No sponsorship can be offered) - Must be SC Cleared or SC Clearable As a Senior Azure AI & Data Consultant you would be designing and delivering next-generation data solutions with Azure Synapse, Azure Databricks, and Microsoft Fabric. This is an...

  • Data Solution Architect

    1 week ago

    England, United Kingdom Cognitive Group | Part of the Focus Cloud Group Full time

    Data Solution Architect - Microsoft Azure Data ServicesMicrosoft Fabrics, Databricks, and Azure AI - PresalesUK (Hybrid) - Must be SC Cleared or eligible for SC Clearance - no sponsorship Permanent What You’ll DoLead the data architecture workstream throughout the pre-sales cycle, helping customers envision innovative, scalable, value-driven...