Data Engineer

2 weeks ago


London, United Kingdom hays-gcj-v4-pd-online Remote Work Freelance Full time

About the RoleWe are seeking a Databricks Data Engineer with strong expertise in designing and optimising large-scale data engineering solutions within the Databricks Data Intelligence Platform. This role is ideal for someone passionate about building high-performance data pipelines and ensuring robust dataernance across modern cloud environments.Key ResponsibilitiesDesign, build, and maintain scalable data pipelines using Databricks Notebooks, Jobs, and Workflows for both batch and streaming data.Optimise Spark and Delta Lake performance through efficient cluster configuration, adaptive query execution, and caching strategies.Conduct performance testing and cluster tuning to ensure cost-efficient, high-performing workloads.Implement data quality, lineage tracking, and access control policies aligned with Databricks Unity Catalogue andernance best practices.Develop PySpark applications for ETL, data transformation, and analytics, following modular and reusable design principles.Create and manage Delta Lake tables with ACIDpliance, schema evolution, and time travel for versioned data management.Integrate Databricks solutions with Azure services such as Azure Data Lake Storage, Key Vault, and Azure Functions.What We’re Looking ForProven experience with Databricks, PySpark, and Delta Lake.Strong understanding of workflow orchestration, performance optimisation, and dataernance.Hands-on experience with Azure cloud services.Ability to work in a fast-paced environment and deliver high-quality solutions.SC Cleared candidates If you're interested in this role, click 'apply now' to forward an up-to-date copy of your CV, or call us now. If this job isn't quite right for you, but you are looking for a new position, please contact us for a confidential discussion about your career. #4750742 - Lottie Cheetham


  • Data Engineer

    5 days ago


    London, United Kingdom Data Idols Full time

    **Data Engineer** Would you like to build enterprise scale data solutions for a global organisation that reaches 400 million users monthly? We are supporting an ambitious data-driven client in the world of Multimedia with their search for a Data Engineer to develop and maintainnew features of their data platform and end-to-end delivery of exciting data...

  • Data Engineer

    2 weeks ago


    London, United Kingdom Data Idols Full time

    **Data Engineer (Azure)** Data Idols are sourcing a Data Engineer for our tech enabled content client in the world of Media. Our client is rapidly scaling, and building a new data team focusing on creating and maintaining a new data environment from scratch - meaning no legacy systemsto deal with! This is a fantastic permanent Data Engineer opportunity,...

  • Data Engineer

    1 day ago


    London, United Kingdom Data Idols Full time

    **Data Engineer** Data Idols are excited to be working with one of the top 100 companies in the world to recruit a Data Engineer! A company which owns multiple brands and who are investing heavily into 'all things data' to revolutionise their extraordinary team. **Skills and Experience**: - Experience with Azure - Experience with SQL - Experience with...

  • Data Engineer

    5 days ago


    London, United Kingdom Data Idols Full time

    **Data Engineer** Data Idols are proud to be working with a growing veterinary business who are extremely data driven and have a great mission to use data to improve the welfare of animals. They have a revolutionary data team who want to be recognised as the UK's leadingveterinary group and are home to some of the best clinicians world-wide; would you like...

  • Data Engineer

    5 days ago


    London, United Kingdom Data Idols Full time

    **Data Engineer** Are you a contractor seeking a new project where you take ownership of Data Science, ETL pipelines, warehouses and everything else analytics, within an alternative and refreshing industry? Our client are a global fashion marketplace, with a mission to praisecreativity, connect consumers to unique and sustainable products, and support...


  • London, United Kingdom Data Idols Full time

    Senior Data Engineer Location: London hybrid Data Idols are working with a high-growth, product-led tech company on a mission to transform a global consumer category. With a rapidly growing customer base and a digital product used by millions, they are scaling their data team to power the next phase of growth and innovation. This role sits at the heart...

  • Azure Data Engineer

    2 weeks ago


    London, United Kingdom Data Idols Full time

    **Azure Data Engineer** Are you an established Data Engineer in the Azure space, looking for your next opportunity? Data Idols are thrilled to be supporting one of the world's largest media platforms with their search for Azure Data Engineers. The business processes billions ofdata points to provide insights, and are welcoming a Data Engineer who has a...

  • Lead Data Engineer

    1 week ago


    London, United Kingdom Data Idols Full time

    **Lead Data Engineer** Are you a Senior or Lead Data Engineer, or perhaps a Data Engineer with more than 4 years experience wanting to up your leadership abilities, and looking for involvement within a nice mix both BAU and greenfield projects? Data Idols are working with a well-known retail company whose brands are certainly well known to households in the...


  • London, United Kingdom Data Idols Full time

    **Senior Data Engineer** Would you like to work for a successful start-up in the Healthtech world? They are ready to welcome their first dedicated Senior Data Engineer into their wider tech team of Data Analysts, reporting to their Head of Data, empowering them to own, shape andgrow their data stack and technologies. **The Opportunity** As the Senior Data...


  • London, United Kingdom Data Idols Full time

    **Data Engineer** If you're a Data Engineer seeking a meaningful new contract opportunity, look no further. We are working with a fast-growing digital HealthTech company, saving the NHS millions per year, and are radically transforming the delivery of health services to makehealthcare work for patients, doctors and other heathcare professionals. They are...