Data Platform Architect

6 days ago

Belfast, United Kingdom Divvy Cloud Corp. Full time
About the Role

The Principal Engineer role is a key part of our Data Platform Engineering team. In this position, you will be focused on helping our product teams move data into our Data Platform for in-product experiences and product analytics.

As a Principal Engineer on the Data Platform Engineering team, you will be responsible for architecting and scaling streaming and batch data pipelines, while also designing the CI/CD infrastructure that ensures efficient development and deployment of data services. You will play a key role in shaping the architecture of our data platform, collaborating with cross-functional teams to deliver highly available, performant, and scalable solutions for both real-time and large-scale data processing.

Key Responsibilities
  • Architect and implement a highly scalable Data Platform that supports Change Data Capture (CDC) using Debezium and Kafka for data replication across different databases and services.
  • Design and maintain large-scale data lakes using Apache Iceberg, ensuring efficient data partitioning, versioning, and schema evolution to support real-time analytics and historical data access.
  • Build and optimize CI/CD pipelines for the deployment and automation of data platform services using tools like Jenkins.
  • Lead the integration of Apache Spark for large-scale data processing and ensure that both batch and streaming workloads are handled efficiently.
  • Collaborate with our Platform Delivery teams to ensure high availability and performance of the data platform, implementing monitoring, disaster recovery, and automated testing frameworks.
  • Ensure that the platform adheres to data governance principles, including data lineage tracking, auditing, and compliance with regulatory requirements.
  • Stay informed about the latest advancements in CDC, data engineering, and infrastructure automation to guide future platform improvements.
  • Work closely with product and data science teams to understand business requirements and translate them into scalable and efficient data platform solutions.
  • Stay current with the latest trends in data engineering and infrastructure, making recommendations for improvements and introducing new technologies as appropriate.
Requirements
  • 10+ years of experience in software engineering with a focus on data platform engineering, data infrastructure, or distributed systems.
  • Expertise in building data pipelines using Apache Kafka or similar for ingesting, processing, and distributing high-throughput data.
  • Strong experience designing and managing CI/CD pipelines for data platform services using tools such as Jenkins.
  • Experience with Apache Iceberg (or similar Delta Lake/Apache Hudi) for managing versioned, partitioned datasets in data lakes with an understanding of Apache Spark for both batch and streaming data processing, including optimization strategies for distributed data workloads.
  • Expertise in designing distributed systems and managing high-throughput, fault-tolerant, and low-latency data architectures.
  • Strong programming skills in Java, Scala, or Python.
  • Experience with cloud-based environments (AWS, GCP, Azure) and containerized infrastructure using Kubernetes and Docker.
  • Strong creative problem-solving skills.

  • Data Platform Architect

    1 week ago

    Belfast, United Kingdom Divvy Cloud Corp. Full time

    About the RoleThe Principal Engineer role is a key part of our Data Platform Engineering team. In this position, you will be responsible for designing and implementing a highly scalable Data Platform that supports Change Data Capture (CDC) using Debezium and Kafka for data replication across different databases and services.You will work closely with our...

  • Data Platform Architect

    1 week ago

    Belfast, United Kingdom Divvy Cloud Corp. Full time

    About the RoleThe Principal Engineer role is a key part of our Data Platform Engineering team. In this position, you will be responsible for designing and implementing a highly scalable Data Platform that supports Change Data Capture (CDC) using Debezium and Kafka for data replication across different databases and services.You will work closely with our...

  • Data Architect

    5 days ago

    Belfast, United Kingdom MCS Group Full time £80,000

    About the RoleThis is an exciting opportunity to join MCS Group as a Data Architect and be part of a team that is shaping the data landscape within a local financial services organisation. As a Data Architect, you will have the chance to work on collaborative Greenfield Data Projects and contribute to the full data lifecycle, including creating the blueprint...

  • Data Architect

    5 days ago

    Belfast, United Kingdom MCS Group Full time £80,000

    About the RoleThis is an exciting opportunity to join MCS Group as a Data Architect and be part of a team that is shaping the data landscape within a local financial services organisation. As a Data Architect, you will have the chance to work on collaborative Greenfield Data Projects and contribute to the full data lifecycle, including creating the blueprint...

  • Data Architect

    3 weeks ago

    Belfast, United Kingdom Ocho Full time

    Job Title: Data ArchitectMy client is a leading organization in the industry, and we are seeking a highly skilled Data Architect to join our team. As a Data Architect, you will play a critical role in designing and implementing data architectures that support business growth and innovation.Key Responsibilities:Design and implement data architectures that...

  • Data Architect

    3 weeks ago

    Belfast, United Kingdom Ocho Full time

    Job Title: Data ArchitectMy client is a leading organization in the industry, and we are seeking a highly skilled Data Architect to join our team. As a Data Architect, you will play a critical role in designing and implementing data architectures that support business growth and innovation.Key Responsibilities:Design and implement data architectures that...

  • Data Architect

    6 days ago

    Belfast, United Kingdom Abacus Professional Recruitment Full time £75,000

    Abacus Careers: Data Architect OpportunityWe are delighted to be working with a large global bank who require a Data Architect to lead their data architecture efforts. If you are interested in learning more about this opportunity, please read on.About the RoleThis is an exciting opportunity to join a team and make a significant impact on the bank's data...

  • Data Architect

    6 days ago

    Belfast, United Kingdom Abacus Professional Recruitment Full time £75,000

    Abacus Careers: Data Architect OpportunityWe are delighted to be working with a large global bank who require a Data Architect to lead their data architecture efforts. If you are interested in learning more about this opportunity, please read on.About the RoleThis is an exciting opportunity to join a team and make a significant impact on the bank's data...

  • Data Architect

    6 days ago

    Belfast, United Kingdom MCS Group Full time £80,000

    Join MCS Group as a Data ArchitectWe are partnering with a leading financial services organization to find a talented Data Architect to drive their digital transformation and change projects.As a Data Architect, you will work closely with a dedicated team of data scientists and engineers to develop and implement a comprehensive data strategy. You will have...

  • Data Architect

    6 days ago

    Belfast, United Kingdom MCS Group Full time £80,000

    Join MCS Group as a Data ArchitectWe are partnering with a leading financial services organization to find a talented Data Architect to drive their digital transformation and change projects.As a Data Architect, you will work closely with a dedicated team of data scientists and engineers to develop and implement a comprehensive data strategy. You will have...

  • Data Architect

    3 weeks ago

    Belfast, United Kingdom MCS Group Full time

    About the RoleMCS Group is partnering with a leading financial services organization to find a talented Data Architect to join their team. This is an exciting opportunity to work on a company-wide digital transformation project, driving data strategy and collaborating with a dedicated team of data scientists and engineers.The successful candidate will have...

  • Data Architect

    3 weeks ago

    Belfast, United Kingdom MCS Group Full time

    About the RoleMCS Group is partnering with a leading financial services organization to find a talented Data Architect to join their team. This is an exciting opportunity to work on a company-wide digital transformation project, driving data strategy and collaborating with a dedicated team of data scientists and engineers.The successful candidate will have...

  • Data Architect

    6 days ago

    Belfast, United Kingdom MCS Group Full time €80,000

    MCS Group is delighted to be partnering with a local financial services organisation as they add a Data Architect to their team in a role that will see you have real impact. The position will see you working on the company wide digital transformation and change projects. You will work with a dedicated team of data scientists and engineers to drive data...

  • Data Architect

    5 days ago

    Belfast, United Kingdom MCS Group Full time €80,000

    MCS Group is delighted to be partnering with a local financial services organisation as they add a Data Architect to their team in a role that will see you have real impact. Candidates should take the time to read all the elements of this job advert carefully Please make your application promptly. The position will see you working on the company wide...

  • Power Platform Solution Architect

    1 month ago

    Belfast, United Kingdom Version 1 Full time

    Job SummaryVersion 1 is seeking a highly skilled Power Platform Solution Architect to join our team. As a key member of our technical team, you will be responsible for designing and implementing complex Power Platform solutions that meet the business needs of our clients.Key ResponsibilitiesCollaborate with clients and internal stakeholders to understand...

  • Power Platform Solution Architect

    1 month ago

    Belfast, United Kingdom Version 1 Full time

    Job SummaryVersion 1 is seeking a highly skilled Power Platform Solution Architect to join our team. As a key member of our technical team, you will be responsible for designing and implementing complex Power Platform solutions that meet the business needs of our clients.Key ResponsibilitiesCollaborate with clients and internal stakeholders to understand...

  • Data Architect

    3 days ago

    Belfast, United Kingdom Laraveldaily Full time

    IT Solutions and Data Architect – Remote OpportunityAs a seasoned IT Solutions and Data Architect, you will play a pivotal role in shaping the company's IT strategy, driving innovation, and delivering cutting-edge solutions. This is a unique opportunity to join a forward-thinking organization that prioritizes sustainable energy solutions and is committed...

  • Data Architect

    2 days ago

    Belfast, United Kingdom Laraveldaily Full time

    IT Solutions and Data Architect – Remote OpportunityAs a seasoned IT Solutions and Data Architect, you will play a pivotal role in shaping the company's IT strategy, driving innovation, and delivering cutting-edge solutions. This is a unique opportunity to join a forward-thinking organization that prioritizes sustainable energy solutions and is committed...

  • Data Solutions Architect

    1 month ago

    Belfast, United Kingdom Kainos Full time

    About the RoleWe are seeking a highly skilled Data Solution Architect to join our team at Kainos. As a key member of our data architecture team, you will be responsible for designing and delivering contemporary data solutions that meet the needs of our clients.Key ResponsibilitiesDesign and implement functional and non-functional designs for data...

  • Data Solutions Architect

    1 month ago

    Belfast, United Kingdom Kainos Full time

    About the RoleWe are seeking a highly skilled Data Solution Architect to join our team at Kainos. As a key member of our data architecture team, you will be responsible for designing and delivering contemporary data solutions that meet the needs of our clients.Key ResponsibilitiesDesign and implement functional and non-functional designs for data...