Data Engineer

2 weeks ago


Leeds, United Kingdom Damia Group Full time

**Spark Scala Engineer – 12 month initial contract – Leeds (3 days in the office) - £550 per day (Inside IR35)**


We are looking for a Spark Scala Engineer to join a long-term project for a management consulting firm working with one of the biggest financial services organisations in the world.

The successful candidate will be responsible for designing, building, and maintaining data pipelines using Apache Spark and Scala. This includes tasks like:

  • Extracting data from various sources (databases, APIs, files)
  • Transforming and cleaning the data
  • Loading the data into data warehouses or data lakes (e.g., BigQuery, Amazon Redshift)
  • Automating the data pipeline execution using scheduling tools (e.g., Airflow)
  • Work with Big Data technologies: You'll likely work with various Big Data technologies alongside Spark, including:

Hadoop Distributed File System (HDFS) for storing large datasets

Apache Kafka for real-time data streaming

Apache Hive for data warehousing on top of HDFS

Cloud platforms like AWS, Azure, or GCP for deploying and managing your data pipelines


Key skills:

  • At least 8+ Years of IT Experience with designing, building, and maintaining data pipelines.
  • At least 4+ Years of experience with designing, building, and maintaining data pipelines using Apache Spark and Scala
  • Programming languages: Proficiency in Scala and Spark is essential. Familiarity with Python and SQL is often a plus.
  • Big Data technologies: Understanding of HDFS, Kafka, Hive, and cloud platforms is valuable.
  • Data engineering concepts: Knowledge of data warehousing, data pipelines, data modelling, and data cleansing techniques is crucial.
  • Problem-solving and analytical skills: You should be able to analyse complex data problems, design efficient solutions, and troubleshoot issues.
  • Communication and collaboration: The ability to communicate effectively with data scientists, analysts, and business stakeholders is essential.


Beneficial skills:

  • Machine learning libraries: Familiarity with Spark ML or other machine learning libraries in Scala can be advantageous.
  • Cloud computing experience: Experience with cloud platforms like AWS, Azure, or GCP for data pipelines deployment is a plus.
  • DevOps tools: Knowledge of DevOps tools like Git, CI/CD pipelines, and containerization tools (Docker, Kubernetes) can be beneficial.



  • Leeds, Leeds, United Kingdom Abs Data Full time

    Job title: Data Centre Engineer – PART TIME Company: Salute Job description : Are you Data Center Facility Engineer looking for a great opportunity but only available part time? Salute has several great opportunities in our key locations – supporting critical facilities – working on a 1 in 3 week call out rota.Salute is a global leader in...

  • Data Engineer

    2 weeks ago


    Leeds, United Kingdom Harnham - Data & Analytics Recruitment Full time

    **DATA ENGINEER** **Coventry 1x week** **Up to £50,000 per annum** **THE COMPANY** This company is one of the largest electrical wholesalers in the UK, which has now branched out to the USA, and is turning over $3 billion per year! They work with businesses and independent customers to provide the highest service and pride themselves ontheir fantastic...


  • Leeds, United Kingdom ON Data Staffing Full time

    We are seeking a dynamic and experienced Data Engnieering professional to join our client, a growing consultancy, as a Data Engineering Manager.As Data Engineering Manager, you will play a pivotal role in driving the success of our client's Data Engineering practice through your expertise in presales, strong technical acumen, consulting skills, and effective...

  • Lead Data Engineer

    1 week ago


    Leeds, Leeds, United Kingdom Harnham - Data & Analytics Recruitment Full time

    Lead Data EngineerUp to £100,000 per annumLocation - Remote - attend office in London once every 3 weeksURGENT HIRE Lead Data Engineer sought for a well-established insurance companyThe CompanyThis company is a fast-growing data-focused general insurance provider. They offer a range of products and services, which include car, bike, van, and home insurance...

  • Senior Data Engineer

    3 weeks ago


    Leeds, United Kingdom Harnham - Data & Analytics Recruitment Full time

    **Senior Data Engineer** **Fully Remote** **Up to £85, 000** **Permanent** A major telecommunications company is looking for a Data Engineer to join them and assist with their migration to AWS. **The Company** An exciting opportunity for a Senior Data Engineer, or an experienced Data Engineer looking for their next career step, has opened up in a...

  • Senior Data Engineer

    3 weeks ago


    Leeds, United Kingdom Harnham - Data & Analytics Recruitment Full time

    **Senior Data Engineer** **Remote - 1 day per month in London per month** **Up to £80, 000** **Permanent** An opportunity for a Senior Data Engineer has opened up at a massive media company, who are looking to build out their APIs and query the datasets. **The Company** An excellent opportunity for a Senior Data Engineer to join a prestigious media...

  • Lead Data Engineer

    4 weeks ago


    Leeds, United Kingdom Harnham - Data & Analytics Recruitment Full time

    **Lead Data Engineer** **Up to £95,000 per annum** **Location** - Fully remote We have a fantastic opportunity for a Lead Data Engineer! **The Company** This client operates within the aviation industry and is one of the UK's leading service providers. They have a network of over 1000 routes across 35 countries. They lead the industry in data, digital,...

  • Head of Data

    3 weeks ago


    Leeds, United Kingdom Harnham - Data & Analytics Recruitment Full time

    **HEAD OF DATA** **REMOTE (MUST be UK based)** **Up to £150,000 + 15% bonus** **THE COMPANY** I'm hiring for a Head of Data to join a B2B food-tech start-up business that uses AI to match food and catering to certain environments (e.g. schools and hospitals). The role reports to the Head of DevOps and the role is a stand-alone, with it being the first...

  • Data Engineer

    1 month ago


    Leeds, United Kingdom Gibbs Hybrid Full time

    **Contract role - Data Engineer - Remote/Leeds/London - Inside IR35 - 8 weeks initial** We are supporting our government organisation customer to look for **.Net Developer**. - Role**:Data Engineer**: - Location**:Remote/Leeds/London**: - Job type**:Contract**: - Duration**:8 weeks initial**: - Data Engineer - Advanced Analytics - Report design and...

  • Data Engineer

    1 week ago


    Leeds, United Kingdom SPG Resourcing Full time

    Salary: 70k Location: Home-based (UK only) with some travel within the UK Start Date: Immediate An innovative Professional Services company is seeking a highly skilled and experienced Senior Azure Data Engineer to join their team and play a pivotal role in their data modernisation journey. Amongst other initiatives, you will be a part of a wider team of...

  • Data Engineer

    2 weeks ago


    Leeds, United Kingdom Harnham Full time

    **DATA ENGINEER** **UP TO £55,000** **REMOTE** **THE COMPANY**: A global information and insights company that handles data from banks across the UK and enables companies to make better financial decisions. **THE ROLE**: The role will involve: - Integrating data from various sources - Cleaning, processing, and transforming data - Designing, building,...

  • Data Engineer

    1 week ago


    Leeds, Leeds, United Kingdom Interface Recruitment UK Full time

    Job Title : Senior Data EngineerThe Role:Chance to work with Cloud technology Big data environment Pioneering technology Truly creative / studio environment Flexbility Leeds City location but with free parking Friendly team with a one for all attitude Summary of position The senior data engineer will be working across our client and SaaS product...

  • Data Engineer

    6 days ago


    Leeds, United Kingdom SPG Resourcing Full time

    An innovative Professional Services company is seeking a highly skilled and experienced Senior Azure Data Engineer to join their team and play a pivotal role in their data modernisation journey.To be considered for an interview, please make sure your application is full in line with the job specs as found below.Amongst other initiatives, you will be a part...

  • Data Engineer

    7 days ago


    Leeds, United Kingdom SPG Resourcing Full time

    An innovative Professional Services company is seeking a highly skilled and experienced Senior Azure Data Engineer to join their team and play a pivotal role in their data modernisation journey.Amongst other initiatives, you will be a part of a wider team of data engineers who can design, build, and maintain robust data pipelines and solutions on the...

  • Data Engineer

    4 weeks ago


    Leeds, United Kingdom Harnham Full time

    **Data Engineer** **Up to £120,000 per annum** **WFH: fully remote** We have a fantastic new opportunity for a data engineer at an exciting new and innovative fintech company! **The Company** This is a remittance company that is currently going through an extraordinary period of hypergrowth and has ambitious growth plans for the next year within its...

  • Data Engineer

    1 month ago


    Leeds, United Kingdom BJSS Full time

    About UsnWe’re an award-winning innovative tech consultancy - a team of creative problem solvers. Since 1993 we’ve been finding better, more sustainable ways to solve complex technology problems for some of the world’s leading organisations and delivered solutions that millions of people use every day.In the last 30 years we won several awards,...

  • Data Engineer

    3 weeks ago


    Leeds, United Kingdom BJSS Full time

    About UsnWe’re an award-winning innovative tech consultancy - a team of creative problem solvers. Since 1993 we’ve been finding better, more sustainable ways to solve complex technology problems for some of the world’s leading organisations and delivered solutions that millions of people use every day.In the last 30 years we won several awards,...

  • Data Engineer

    1 week ago


    Leeds, Leeds, United Kingdom Fruition IT Full time

    Job DescriptionData EngineerLeeds - on site 2x per weekup to £55,000Looking to join a dynamic team in the financial sector? Fruition IT are looking for an experienced Data Engineer to join a leading company specializing in financial solutions.Working closely with the Senior Data Engineer and Architects, you will play a crucial role in their journey of...

  • Data Engineer

    1 month ago


    Leeds, United Kingdom BJSS Full time

    About UsWe’re an award-winning innovative tech consultancy - a team of creative problem solvers. Since 1993 we’ve been finding better, more sustainable ways to solve complex technology problems for some of the world’s leading organisations and delivered solutions that millions of people use every day.In the last 30 years we won several awards,...

  • Data Engineer

    1 month ago


    Leeds, United Kingdom BJSS Full time

    About Us We’re an award-winning innovative tech consultancy - a team of creative problem solvers. Since 1993 we’ve been finding better, more sustainable ways to solve complex technology problems for some of the world’s leading organisations and delivered solutions that millions of people use every day. In the last 30 years we won several awards,...