SC Cleared Databricks Data Engineer – Azure Cloud

5 days ago

England, United Kingdom Montash Full time

Job Title: SC Cleared Databricks Data Engineer – Azure Cloud Contract Type: 12 month contract Day Rate: Up to £400 a day inside IR35 Location: Remote or hybrid (as agreed) Start Date: January 5th 2026 Clearance required: Must be holding active SC Clearance We are seeking an experienced Databricks Data Engineer to design, build, and optimise large-scale data workflows within the Databricks Data Intelligence Platform. The role focuses on delivering high-performing batch and streaming pipelines using PySpark, Delta Lake, and Azure services, with additional emphasis on governance, lineage tracking, and workflow orchestration. Client information remains confidential. Key Responsibilities Build and orchestrate Databricks data pipelines using Notebooks, Jobs, and Workflows Optimise Spark and Delta Lake workloads through cluster tuning, adaptive execution, scaling, and caching Conduct performance benchmarking and cost optimisation across workloads Implement data quality, lineage, and governance practices aligned with Unity Catalog Develop PySpark-based ETL and transformation logic using modular, reusable coding standards Create and manage Delta Lake tables with ACID compliance, schema evolution, and time travel Integrate Databricks assets with Azure Data Lake Storage, Key Vault, and Azure Functions Collaborate with cloud architects, data analysts, and engineering teams on end-to-end workflow design Support automated deployment of Databricks artefacts via CI/CD pipelines Maintain clear technical documentation covering architecture, performance, and governance configuration Required Skills and Experience Strong experience with the Databricks Data Intelligence Platform Hands-on experience with Databricks Jobs and Workflows Deep PySpark expertise, including schema management and optimisation Strong understanding of Delta Lake architecture and incremental design principles Proven Spark performance engineering and cluster tuning capabilities Unity Catalog experience (data lineage, access policies, metadata governance) Azure experience across ADLS Gen2, Key Vault, and serverless components Familiarity with CI/CD deployment for Databricks Solid troubleshooting skills in distributed environments Preferred Qualifications Experience working across multiple Databricks workspaces and governed catalogs Knowledge of Synapse, Power BI, or related Azure analytics services Understanding of cost optimisation for data compute workloads Strong communication and cross-functional collaboration skills

  • SC Cleared Databricks Data Engineer

    3 days ago

    England, United Kingdom Montash Full time

    Job Title: SC Cleared Databricks Data Engineer – Azure Cloud Contract Type: 12 month contract Day Rate: Up to £400 a day inside IR35 Location: Remote or hybrid (as agreed) Start Date: January 5th 2026 Clearance required: Must be holding active SC Clearance We are seeking an experienced Databricks Data Engineer to design, build, and optimise large-scale...

  • SC Cleared Databricks Data Engineer – Azure Cloud

    5 days ago

    England, United Kingdom Montash Full time

    Job Title: SC Cleared Databricks Data Engineer – Azure CloudContract Type: 12 month contractDay Rate: Up to £400 a day inside IR35Location: Remote or hybrid (as agreed)Start Date: January 5th 2026Clearance required: Must be holding active SC ClearanceWe are seeking an experienced Databricks Data Engineer to design, build, and optimise large-scale data...

  • Databricks Engineer

    4 weeks ago

    England, United Kingdom iO Associates Full time

    Databricks Engineer - SC Cleared Fully Remote Duration: 6 Months Location: London Rate: £500-£550 per day (Inside IR35) Start Date: ASAP Are you an SC Cleared Databricks Engineer with a passion for building scalable data solutions? Do you have extensive experience in designing and optimising data pipelines, cloud-native platforms, and advanced analytics...

  • Azure Cloud Data Engineer

    3 days ago

    England, United Kingdom Montash Full time

    Job Title: SC Cleared Databricks Data Engineer – Azure Cloud Contract Type: 12 month contract Day Rate: Up to £400 a day inside IR35 Location: Remote or hybrid (as agreed) Start Date: January 5th 2026 Clearance required: Must be holding active SC Clearance We are seeking an experienced Databricks Data Engineer to design, build, and optimise...

  • SC cleared

    2 weeks ago

    England, United Kingdom Methods Full time

    Duration: initial contract until end of March 26, with possible extensionDaily rate : £495 inside IR35Location: RemoteClearance: active SC Key Responsibilities:Design, develop, and optimize data pipelines using Microsoft Fabric and Azure services (eg, Azure Data Factory, Azure Synapse Analytics, Azure Databricks).Build and maintain scalable,...

  • SC cleared

    4 weeks ago

    England, United Kingdom Methods Full time

    Duration: initial contract until end of March 26, with possible extension Daily rate : £495 inside IR35 Location: Remote Clearance : active SC Key Responsibilities: Design, develop, and optimize data pipelines using Microsoft Fabric and Azure services (eg, Azure Data Factory, Azure Synapse Analytics, Azure Databricks). Build and maintain scalable,...

  • Senior Data

    3 weeks ago

    england, United Kingdom Cognitive Group | Part of the Focus Cloud Group Full time

    Senior AI & Data Consultant - Microsoft Fabric Permanent - Fulltime Homebased - Hybrid working in the UK (No sponsorship can be offered) - Must be SC Cleared or SC Clearable As a Senior Azure AI & Data Consultant you would be designing and delivering next-generation data solutions with Azure Synapse , Azure Databricks , and Microsoft Fabric . This is an...

  • Senior Data

    3 days ago

    England, United Kingdom Cognitive Group | Part of the Focus Cloud Group Full time

    Senior AI & Data Consultant - Microsoft Fabric Permanent - Fulltime Homebased - Hybrid working in the UK (No sponsorship can be offered) - Must be SC Cleared or SC Clearable As a Senior Azure AI & Data Consultant you would be designing and delivering next-generation data solutions with Azure Synapse , Azure Databricks , and Microsoft Fabric . This is an...

  • Senior Data

    3 days ago

    England, United Kingdom Cognitive Group | Part of the Focus Cloud Group Full time

    Senior AI & Data Consultant - Microsoft FabricPermanent - FulltimeHomebased - Hybrid working in the UK(No sponsorship can be offered) - Must be SC Cleared or SC Clearable As a Senior Azure AI & Data Consultant you would be designing and delivering next-generation data solutions with Azure Synapse, Azure Databricks, and Microsoft Fabric. This is an...

  • Data Solution Architect

    1 week ago

    England, United Kingdom Cognitive Group | Part of the Focus Cloud Group Full time

    Data Solution Architect - Microsoft Azure Data ServicesMicrosoft Fabrics, Databricks, and Azure AI - PresalesUK (Hybrid) - Must be SC Cleared or eligible for SC Clearance - no sponsorship Permanent What You’ll DoLead the data architecture workstream throughout the pre-sales cycle, helping customers envision innovative, scalable, value-driven...