Data Engineering Lead for Large-Scale Data Systems

5 days ago


Belfast, United Kingdom Kainos Full time

Discover a challenging opportunity as a Senior Data Engineer at Kainos, where you will design and develop data processing software components for solutions handling large datasets.

We are seeking an experienced professional to join our agile team, providing strong development leadership and taking responsibility for significant technical components of data systems. Your primary focus will be on developing large-scale data processing software to meet user needs in demanding production environments.

Main Responsibilities:

  • Design and develop data processing software primarily for deployment in Big Data technologies.
  • Work with Architects and Lead Engineers to ensure the software supports non-functional needs.
  • Collaborate with colleagues to resolve implementation challenges and maintain high code quality.
  • Ensure operational readiness with operations teams.
  • Advise customers and managers on estimated effort and technical implications of user stories.
  • Coach and mentor team members.

You will have the opportunity to work on complex projects, applying your expertise in software development, data processing platforms, and distributed data processing frameworks. Our ideal candidate will possess strong analytical skills, attention to detail, and excellent communication skills.

Mandatory Requirements:

  • Strong software development experience in Java, Scala, or Python.
  • Experience with data-processing platforms from vendors such as AWS, Azure, GCP, Databricks.
  • Substantial experience in developing large-scale data processing solutions and deploying into production environments.
  • Proficient in SQL and SQL extensions for analytical queries.
  • Solid understanding of ETL/ELT data processing pipelines and design patterns.
  • Aware of key features and pitfalls of distributed data processing frameworks, data stores, and data serialization formats.
  • Ability to write quality, testable code and experience with automated testing.
  • Experience with Continuous Integration and Continuous Deployment techniques.

The salary range for this position is $120,000 - $180,000 per year, based on location and experience. Benefits include comprehensive health insurance, retirement plan, paid time off, and professional development opportunities. As part of our commitment to diversity and inclusion, we actively seek out talented individuals from all backgrounds and encourage applications from underrepresented groups.


  • Data Engineer

    5 days ago


    Belfast, United Kingdom Data Intellect Full time

    Job OverviewWe are seeking a highly skilled Data Engineer to join our team at Data Intellect. As a Data Engineer, you will be responsible for designing, developing, and implementing data-driven solutions that transform our clients' businesses.Key ResponsibilitiesDesign and Develop Data-Driven SolutionsCollaborate with Cross-Functional TeamsImplement and...

  • Data Engineer

    4 weeks ago


    Belfast, United Kingdom Data Intellect Full time

    Job DescriptionWhat you’ll be doing:Build and enhance bespoke Data Science and Engineering solutions for our clientsEnhance and grow your data science and engineering knowledge among subject matter experts in our learning and development ecosystem Deliver high quality, best practice software solutions Use a range of architectures, tools and...


  • Belfast, United Kingdom Data Intellect Full time

    Job DescriptionWe are seeking a highly skilled Data Scientist and Engineer to join our team at Data Intellect. As a key member of our data science and engineering team, you will be responsible for designing, developing, and implementing data-driven solutions for our clients.Key Responsibilities:Design and develop bespoke data science and engineering...


  • Belfast, United Kingdom Version 1 Full time

    Job DescriptionWe are seeking a highly skilled Data Engineer to join our team at Version 1. The successful candidate will be responsible for designing and implementing data ingestion and transformation pipelines for large-scale organisations. This is an exciting opportunity to work with cutting-edge technologies and contribute to the development of...

  • Lead Data Engineer

    5 days ago


    Belfast, United Kingdom Divvy Cloud Corp. Full time

    Job SummaryWe are seeking a Lead Data Engineer to join our Data Platform Engineering team at Divvy Cloud Corp. As a key member of the team, you will be responsible for leading the design and implementation of scalable data pipelines using Apache Kafka, Apache Iceberg, and Apache Spark.You will work closely with cross-functional teams to deliver highly...


  • Belfast, United Kingdom Data Intellect Full time

    Job Title: Databricks Practice LeadJob Summary:Data Intellect is seeking a highly skilled Databricks Practice Lead to drive the growth, innovation, and successful delivery of cloud-based data solutions for our clients. As a key member of our team, you will collaborate with internal teams and clients to design, implement, and optimize big data and machine...


  • Belfast, United Kingdom Ocho Full time

    Staff Data EngineerOcho is seeking a highly skilled Staff Data Engineer to join our team. This role focuses on enabling brands and retailers to gather, analyze, and act on valuable user-generated content at a massive scale.Main Responsibilities:* Design, build, and maintain large-scale, distributed data systems that support the company's infrastructure and...

  • Senior Data Engineer

    4 weeks ago


    Belfast, United Kingdom Rapid7 Full time

    About the RoleRapid7 is seeking a highly skilled Data Platform Engineer to join our team. As a key member of our Data Platform team, you will be responsible for designing, developing, and operating highly reliable large-scale data lake systems.Key ResponsibilitiesAnalyse and optimise distributed SQL queries to improve performanceSuggest optimisations to our...


  • Belfast, United Kingdom Search 5.0 Full time €75,000 - €90,000

    Lead Data Engineer – Belfast/Hybrid - £75 – 90k They tackle co mplex challenges, innovate, and achieve results through a blend of strategy, technology, and expertise . Their work spans a wide range of industries, including healthcare, energy, government, financial services, and consumer products. Their Belfast office is a hub of innovation ...


  • Belfast, United Kingdom Search 5.0 Full time €75,000 - €90,000

    Lead Data Engineer – Belfast/Hybrid - £75 – 90k They tackle co mplex challenges, innovate, and achieve results through a blend of strategy, technology, and expertise . Their work spans a wide range of industries, including healthcare, energy, government, financial services, and consumer products. Their Belfast office is a hub of innovation ...


  • Belfast, United Kingdom Divvy Cloud Corp. Full time

    About the RoleThe Data Platform Engineering team at Divvy Cloud Corp. is seeking a highly skilled Senior Data Platform Engineer to join our team. As a Senior Data Platform Engineer, you will be responsible for architecting and scaling streaming and batch data pipelines, while also designing the CI/CD infrastructure that ensures efficient development and...

  • Senior Data Engineer

    4 weeks ago


    Belfast, United Kingdom Rapid7 Full time

    As a key member of the Rapid7 Data Platform Engineering team, you'll have the opportunity to design and implement a highly scalable Data Platform that supports Change Data Capture (CDC) using Debezium and Kafka for data replication across different databases and services. About the Role The Data Platform team is responsible for building ETL pipelines that...


  • Belfast, United Kingdom MCS Group Full time

    **Job Summary**MCS Group is seeking a highly skilled Civil Engineering Data Management Lead to join our team in Belfast. As a key member of our organization, you will be responsible for overseeing the management of data and civil engineering projects, ensuring seamless delivery from initiation to completion.**About the Role**The successful candidate will...

  • Lead Data Engineer

    4 months ago


    Belfast, United Kingdom PA Consulting Full time

    Job DescriptionAre you up for the challenge of creating seamless user experiences from every angle?Our teams are trusted to deliver and given the space to be awesome. We’re an inclusive community for the curious, generous, pragmatic and committed digital practitioner. You'll be leading teams who are collectively responsible for delivering functioning...


  • Belfast, United Kingdom Search 5.0 Full time

    Lead Data Engineer – Belfast/Hybrid - £75 – 90k They tackle co mplex challenges, innovate, and achieve results through a blend of strategy, technology, and expertise . Their work spans a wide range of industries, including healthcare, energy, government, financial services, and consumer products. Their Belfast office is a hub of innovation –...

  • Data Engineer

    3 months ago


    Belfast, United Kingdom Rapid7 Full time

    Data Engineer II Rapid7 seeks a highly motivated and inquisitive aspiring Data Engineer II to join our quickly scaling data engineering function. Come and join our efforts in unlocking the value of data through industry-leading innovation, cutting edge modern tooling, democratization at scale and building exceptional and trusted data products for the...

  • Lead Data Engineer

    1 day ago


    Belfast, United Kingdom Search 5.0 Full time

    Lead Data Engineer – Belfast/Hybrid - £75 – 90kThis company is one of Belfast’s best kept secrets. They tackle complex challenges, innovate, and achieve results through a blend of strategy, technology, and expertise. Their work spans a wide range of industries, including healthcare, energy, government, financial services, and consumer products.Their...

  • Lead Data Engineer

    15 hours ago


    Belfast, United Kingdom Search 5.0 Full time

    Lead Data Engineer – Belfast/Hybrid - £75 – 90k This company is one of Belfast’s best kept secrets. They tackle co mplex challenges, innovate, and achieve results through a blend of strategy, technology, and expertise . Their work spans a wide range of industries, including healthcare, energy, government, financial services, and consumer...

  • Lead Data Engineer

    1 day ago


    Belfast, United Kingdom Search 5.0 Full time

    Lead Data Engineer – Belfast/Hybrid - £75 – 90kThis company is one of Belfast’s best kept secrets. They tackle complex challenges, innovate, and achieve results through a blend of strategy, technology, and expertise. Their work spans a wide range of industries, including healthcare, energy, government, financial services, and consumer products.Their...

  • Lead Data Engineer

    1 day ago


    Belfast, United Kingdom Search 5.0 Full time €75,000 - €90,000

    Lead Data Engineer – Belfast/Hybrid - £75 – 90k This company is one of Belfast’s best kept secrets. They tackle co mplex challenges, innovate, and achieve results through a blend of strategy, technology, and expertise . Their work spans a wide range of industries, including healthcare, energy, government, financial services, and consumer...