Azure Cloud Data Engineer

3 days ago

England, United Kingdom Montash Full time

Job Title: SC Cleared Databricks Data Engineer – Azure Cloud Contract Type: 12 month contract
Day Rate: Up to £400 a day inside IR35
Location: Remote or hybrid (as agreed)
Start Date: January 5th 2026
Clearance required: Must be holding active SC Clearance

We are seeking an experienced Databricks Data Engineer to design, build, and optimise large-scale data workflows within the Databricks Data Intelligence Platform.

The role focuses on delivering high-performing batch and streaming pipelines using PySpark, Delta Lake, and Azure services, with additional emphasis on governance, lineage tracking, and workflow orchestration. Client information remains confidential.

Key Responsibilities
Build and orchestrate Databricks data pipelines using Notebooks, Jobs, and Workflows
Optimise Spark and Delta Lake workloads through cluster tuning, adaptive execution, scaling, and caching
Conduct performance benchmarking and cost optimisation across workloads
Implement data quality, lineage, and governance practices aligned with Unity Catalog
Develop PySpark-based ETL and transformation logic using modular, reusable coding standards
Create and manage Delta Lake tables with ACID compliance, schema evolution, and time travel
Integrate Databricks assets with Azure Data Lake Storage, Key Vault, and Azure Functions
Collaborate with cloud architects, data analysts, and engineering teams on end-to-end workflow design
Support automated deployment of Databricks artefacts via CI/CD pipelines
Maintain clear technical documentation covering architecture, performance, and governance configuration

Required Skills and Experience
Strong experience with the Databricks Data Intelligence Platform
Hands-on experience with Databricks Jobs and Workflows
Deep PySpark expertise, including schema management and optimisation
Strong understanding of Delta Lake architecture and incremental design principles
Proven Spark performance engineering and cluster tuning capabilities
Unity Catalog experience (data lineage, access policies, metadata governance)
Azure experience across ADLS Gen2, Key Vault, and serverless components
Familiarity with CI/CD deployment for Databricks
Solid troubleshooting skills in distributed environments

Preferred Qualifications
Experience working across multiple Databricks workspaces and governed catalogs
Knowledge of Synapse, Power BI, or related Azure analytics services
Understanding of cost optimisation for data compute workloads
Strong communication and cross-functional collaboration skills

  • Azure Cloud Engineer

    1 week ago

    England, United Kingdom Tenth Revolution Group Full time

    🌐 Job Ad: Azure Cloud Engineer (Legal Experience Required) – Outside IR35 – £500/dayLocation: Hybrid – 2 days per week onsite at a range of UK offices (London, Manchester, Birmingham, Leeds) Contract: 6 months initial (likely extension) Rate: £500/day (Outside IR35)About the RoleWe are seeking an experienced Azure Cloud Engineer with proven legal...

  • Azure Cloud Engineer

    1 week ago

    England, United Kingdom Tenth Revolution Group Full time

    🌐 Job Ad: Azure Cloud Engineer (Legal Experience Required) – Outside IR35 – £500/dayLocation: Hybrid – 2 days per week onsite at a range of UK offices (London, Manchester, Birmingham, Leeds) Contract: 6 months initial (likely extension) Rate: £500/day (Outside IR35)About the RoleWe are seeking an experienced Azure Cloud Engineer with proven legal...

  • Azure Cloud Engineer

    1 week ago

    England, United Kingdom Tenth Revolution Group Full time

    Job Ad: Azure Cloud Engineer (Legal Experience Required) – Outside IR35 – £500/dayLocation: Hybrid – 2 days per week onsite at a range of UK offices (London, Manchester, Birmingham, Leeds) Contract: 6 months initial (likely extension) Rate: £500/day (Outside IR35)About the RoleWe are seeking an experienced Azure Cloud Engineer with proven legal...

  • Data Engineer

    1 week ago

    London|South East (England), United Kingdom Cloud Decisions Full time £60,000 - £90,000 per year

    Data Engineer - Manager x5There are data teams and then there are data teams Our customer puts enterprise at a new scale, not only with the size of their client engagements, but also with their own size as one of the big 4 and one of the first ever Microsoft Global Alliance Partners (*all data technologies will be delivered (GCP/AWS/Big data) although...

  • SC Cleared Databricks Data Engineer – Azure Cloud

    5 days ago

    England, United Kingdom Montash Full time

    Job Title: SC Cleared Databricks Data Engineer – Azure Cloud Contract Type: 12 month contract Day Rate: Up to £400 a day inside IR35 Location: Remote or hybrid (as agreed) Start Date: January 5th 2026 Clearance required: Must be holding active SC Clearance We are seeking an experienced Databricks Data Engineer to design, build, and optimise large-scale...

  • SC Cleared Databricks Data Engineer

    3 days ago

    England, United Kingdom Montash Full time

    Job Title: SC Cleared Databricks Data Engineer – Azure Cloud Contract Type: 12 month contract Day Rate: Up to £400 a day inside IR35 Location: Remote or hybrid (as agreed) Start Date: January 5th 2026 Clearance required: Must be holding active SC Clearance We are seeking an experienced Databricks Data Engineer to design, build, and optimise large-scale...

  • SC Cleared Databricks Data Engineer – Azure Cloud

    5 days ago

    England, United Kingdom Montash Full time

    Job Title: SC Cleared Databricks Data Engineer – Azure CloudContract Type: 12 month contractDay Rate: Up to £400 a day inside IR35Location: Remote or hybrid (as agreed)Start Date: January 5th 2026Clearance required: Must be holding active SC ClearanceWe are seeking an experienced Databricks Data Engineer to design, build, and optimise large-scale data...

  • Azure Cloud Engineer

    1 week ago

    City Of London, England, United Kingdom Selby Jennings Full time £100,000 - £120,000 per year

    Our client, a fast-growing quantitative trading firm, is seeking a hands-on Cloud Engineer to design, build, and maintain robust cloud infrastructure in a high-performance, security-sensitive environment. This is an opportunity to work at the intersection of cutting-edge technology and finance, ensuring systems are reliable, efficient, and compliant in a...

  • Director of Azure Engineering

    1 day ago

    England, United Kingdom Syniti Full time

    Syniti, part of Capgemini, tackles the hardest work in data for the world’s largest organizations. We combine intelligent software with deep data expertise to help the Fortune2000 tackle complex data challenges and drive measurable business outcomes with business-ready data. Syniti’s Data First strategy transforms data from an afterthought into a...

  • Senior Data

    3 days ago

    England, United Kingdom Cognitive Group | Part of the Focus Cloud Group Full time

    Senior AI & Data Consultant - Microsoft FabricPermanent - FulltimeHomebased - Hybrid working in the UK(No sponsorship can be offered) - Must be SC Cleared or SC Clearable As a Senior Azure AI & Data Consultant you would be designing and delivering next-generation data solutions with Azure Synapse, Azure Databricks, and Microsoft Fabric. This is an...