Principal Software Architect

1 week ago

Belfast, United Kingdom Rapid7 Full time

As a Principal Software Architect at Rapid7, you will have the opportunity to design and implement a highly scalable Data Platform that supports Change Data Capture (CDC) using Debezium and Kafka for data replication across different databases and services.

About the Role

The Principal Software Architect role is a part of our Data Platform Engineering team. In this role, you will be responsible for architecting and scaling streaming and batch data pipelines, while also designing the CI/CD infrastructure that ensures efficient development and deployment of data services.

You will play a key role in shaping the architecture of our data platform, collaborating with cross-functional teams to deliver highly available, performant, and scalable solutions for both real-time and large-scale data processing.

Key Responsibilities

  • Architect and implement a highly scalable Data Platform that supports Change Data Capture (CDC) using Debezium and Kafka for data replication across different databases and services.
  • Design and maintain large-scale data lakes using Apache Iceberg, ensuring efficient data partitioning, versioning, and schema evolution to support real-time analytics and historical data access.
  • Build and optimize CI/CD pipelines for the deployment and automation of data platform services using tools like Jenkins.
  • Lead the integration of Apache Spark for large-scale data processing and ensure that both batch and streaming workloads are handled efficiently.
  • Collaborate with our Platform Delivery teams to ensure high availability and performance of the data platform, implementing monitoring, disaster recovery, and automated testing frameworks.
  • Provide technical leadership and mentoring to junior engineers, promoting best practices in CDC architecture, distributed systems, and CI/CD automation.
  • Ensure that the platform adheres to data governance principles, including data lineage tracking, auditing, and compliance with regulatory requirements.
  • Stay informed about the latest advancements in CDC, data engineering, and infrastructure automation to guide future platform improvements.
  • Work closely with product and data science teams to understand business requirements and translate them into scalable and efficient data platform solutions.
  • Stay current with the latest trends in data engineering and infrastructure, making recommendations for improvements and introducing new technologies as appropriate.

Requirements

  • 10+ years of experience in software engineering with a focus on data platform engineering, data infrastructure, or distributed systems.
  • Expertise in building data pipelines using Apache Kafka or similar for ingesting, processing, and distributing high-throughput data.
  • Strong experience designing and managing CI/CD pipelines for data platform services using tools such as Jenkins.
  • Experience with Apache Iceberg (or similar Delta Lake/Apache Hudi) for managing versioned, partitioned datasets in data lakes with an understanding of Apache Spark for both batch and streaming data processing, including optimization strategies for distributed data workloads.
  • Expertise in designing distributed systems and managing high-throughput, fault-tolerant, and low-latency data architectures.
  • Strong programming skills in Java, Scala, or Python.
  • Experience with cloud-based environments (AWS, GCP, Azure) and containerized infrastructure using Kubernetes and Docker.
  • The attitude and ability to thrive in a high-growth, evolving environment.
  • Collaborative team player who has the ability to partner with others and drive toward solutions.
  • Strong creative problem-solving skills.
  • Solid communicator with excellent written and verbal communications skills both within the team and cross-functionally.
  • Passionate about delighting customers, puts the customer needs at the forefront of all decision-making.
  • Excellent attention to detail.

  • Principal Software Architect

    4 weeks ago

    Belfast, United Kingdom TP ICAP Full time

    Job Title: Principal Engineer Job Summary: TP ICAP is seeking a highly skilled Principal Engineer to join our team. As a Principal Engineer, you will provide technical leadership and guidance to our development teams, ensuring the delivery of high-quality software solutions. You will be responsible for designing, developing, and overseeing the production...

  • Principal Software Architect

    4 weeks ago

    Belfast, United Kingdom TP ICAP Full time

    Job Title: Principal Engineer Job Summary: TP ICAP is seeking a highly skilled Principal Engineer to join our team. As a Principal Engineer, you will provide technical leadership and guidance to our development teams, ensuring the delivery of high-quality software solutions. You will be responsible for designing, developing, and overseeing the production...

  • Principal Software Architect

    1 week ago

    Belfast, United Kingdom Rapid7 Full time

    About the RoleThe Principal Software Architect will be responsible for architecting and scaling streaming and batch data pipelines, while also designing the CI/CD infrastructure that ensures efficient development and deployment of data services. This role will play a key part in shaping the architecture of our data platform, collaborating with...

  • Principal Software Architect

    3 weeks ago

    Belfast, United Kingdom MCS Group Full time

    Exciting Opportunity for a Principal Software EngineerMCS Group is proud to be collaborating with a leading global financial services firm based in Belfast as they seek to hire a Principal Software Engineer specializing in Front End development. This organization is at the forefront of their industry worldwide and takes pride in having a skilled technology...

  • Principal Software Architect

    4 weeks ago

    Belfast, United Kingdom MCS Group Full time

    Exciting Opportunity for a Principal Software EngineerMCS Group is proud to be collaborating with a leading global financial services firm based in Belfast as they seek to hire a Principal Software Engineer specializing in Front End development. This organization is a pioneer in their industry worldwide and takes pride in having a skilled technology hub...

  • Principal Software Architect

    4 weeks ago

    Belfast, United Kingdom MCS Group Full time

    Exciting Opportunity for a Principal Software EngineerMCS Group is proud to be collaborating with a leading global financial services firm based in Belfast as they seek to hire a Principal Software Engineer specializing in Front End development. This organization is a pioneer in their industry worldwide and takes pride in having a skilled technology hub...

  • Principal Software Architect

    3 weeks ago

    Belfast, United Kingdom MCS Group Full time

    Exciting Opportunity for a Principal Software EngineerMCS Group is proud to be collaborating with a leading global financial services firm based in Belfast as they seek to hire a Principal Software Engineer specializing in Front End development. This organization is at the forefront of their industry worldwide and takes pride in having a skilled technology...

  • Principal Software Architect

    3 weeks ago

    Belfast, United Kingdom MCS Group Full time

    Exciting Opportunity for a Principal Software EngineerMCS Group is proud to be collaborating with a leading global financial services firm based in Belfast as they seek to hire a Principal Software Engineer specializing in Front End development. This organization is a pioneer in their industry worldwide and takes pride in having a skilled technology hub...

  • Principal Software Architect

    3 weeks ago

    Belfast, United Kingdom MCS Group Full time

    Exciting Opportunity for a Principal Software EngineerMCS Group is proud to be collaborating with a leading global financial services firm based in Belfast as they seek to hire a Principal Software Engineer specializing in Front End development. This organization is a pioneer in their industry worldwide and takes pride in having a skilled technology hub...

  • Principal Software Architect

    4 weeks ago

    Belfast, United Kingdom TP ICAP Full time

    Job Title: Principal EngineerThe TP ICAP Group is a leading provider of market infrastructure solutions. Our purpose is to deliver innovative and responsible solutions that improve price discovery, liquidity, and data distribution for our clients. We achieve this through our people and technology, connecting clients to superior liquidity and data...

  • Principal Software Architect

    4 weeks ago

    Belfast, United Kingdom TP ICAP Full time

    Job Title: Principal EngineerThe TP ICAP Group is a leading provider of market infrastructure solutions. Our purpose is to deliver innovative and responsible solutions that improve price discovery, liquidity, and data distribution for our clients. We achieve this through our people and technology, connecting clients to superior liquidity and data...

  • Principal Software Architect

    4 weeks ago

    Belfast, United Kingdom Rapid7 Full time

    About the RoleThe Principal Engineer position is a key part of our Data Platform Engineering team at Rapid7. In this role, you will be responsible for designing and implementing a highly scalable Data Platform that supports Change Data Capture (CDC) using Debezium and Kafka for data replication across different databases and services.Key...

  • Principal Software Architect

    4 weeks ago

    Belfast, United Kingdom Rapid7 Full time

    About the RoleThe Principal Engineer position is a key part of our Data Platform Engineering team at Rapid7. In this role, you will be responsible for designing and implementing a highly scalable Data Platform that supports Change Data Capture (CDC) using Debezium and Kafka for data replication across different databases and services.Key...

  • Principal Software Engineer

    2 weeks ago

    Belfast, United Kingdom Adepta Partners Limited Full time

    Principal .NET EngineerWe are seeking a highly skilled Principal .NET Engineer to lead the development of robust, high-performance applications on the .NET Core platform for one of our rapidly growing partners.Key Responsibilities:Design and develop scalable solutions using .NET Core, with a focus on REST APIs and messaging systems like Kafka.Lead and...

  • Principal Network Architect

    3 weeks ago

    Belfast, United Kingdom Adepta Partners Full time £100,000

    Network Engineer Job DescriptionSuitable for Principal/Staff Network Engineer or ArchitectAdepta Partners is a team of talented individuals driven to deliver solutions that are responsive to the real world, empowered with the freedom to get creative about how they do it. Our company ethos is one of inclusivity, flexibility, and community.We're looking for an...

  • Principal Network Architect

    3 weeks ago

    Belfast, United Kingdom Adepta Partners Full time £100,000

    Network Engineer Job DescriptionSuitable for Principal/Staff Network Engineer or ArchitectAdepta Partners is a team of talented individuals driven to deliver solutions that are responsive to the real world, empowered with the freedom to get creative about how they do it. Our company ethos is one of inclusivity, flexibility, and community.We're looking for an...

  • Principal Machine Learning Architect

    1 month ago

    Belfast, United Kingdom Ocho Full time

    Job Title: Principal Machine Learning ArchitectWe are seeking a seasoned Principal Machine Learning Architect to lead our dynamic team of skilled ML Engineers. As a key member of our Machine Learning Engineering team, you will collaborate with the Engineering Director to design and deploy scalable, high-performance ML solutions.Key...

  • Principal Machine Learning Architect

    1 month ago

    Belfast, United Kingdom Ocho Full time

    Job Title: Principal Machine Learning ArchitectWe are seeking a seasoned Principal Machine Learning Architect to lead our dynamic team of skilled ML Engineers. As a key member of our Machine Learning Engineering team, you will collaborate with the Engineering Director to design and deploy scalable, high-performance ML solutions.Key...

  • Senior Software Architect

    7 days ago

    Belfast, United Kingdom Johnson Controls Full time

    Job Title: Principal Software EngineerAbout the Role:As a Principal Software Engineer at Johnson Controls, you will be part of our global team that creates innovative, integrated solutions to make people, facilities, and assets safe with our industry-leading security solutions. This is an exciting opportunity to develop the next generation of AI-driven...

  • Chief Software Architect

    4 weeks ago

    Belfast, United Kingdom Divvy Cloud Corp. Full time

    About the RoleThe Principal Software Engineer is a key member of our Product and Engineering team, responsible for designing and developing secure and scalable software solutions. In this role, you will work closely with our customers to understand their security needs and develop innovative solutions to address them.Key ResponsibilitiesDesign and develop...