Data Engineering Professional for Large-Scale Organisations
1 week ago
We are seeking a highly skilled Data Engineer to join our team at Version 1. The successful candidate will be responsible for designing and implementing data ingestion and transformation pipelines for large-scale organisations. This is an exciting opportunity to work with cutting-edge technologies and contribute to the development of innovative solutions.
Key Responsibilities:
- Designing and implementing high-performing data ingestion and transformation pipelines from multiple sources using various technologies.
- Delivering and presenting proofs of concept of key technology components to prospective customers and project stakeholders.
- Developing scalable and reusable frameworks for ingestion and transformation of large data sets.
- Mastery of data management system and process design, as well as implementation.
- Data quality system and process design and implementation.
- Integrating end-to-end data pipelines to ensure data quality and consistency.
- Working with event-based/streaming technologies to ingest and process data.
- Collaborating with other project team members to support delivery of additional project components (Reporting tools, API interfaces, Search).
- Evaluating performance and applicability of multiple tools against customer requirements.
- Working within an Agile delivery/DevOps methodology to deliver proof of concept and production implementation in iterative sprints.
Requirements and Qualifications:
Azure Data Analytics Platform Experience:
- Hands-on experience designing and delivering solutions using the Azure Data Analytics platform (Cortana Intelligence Platform) including Azure Storage, Azure SQL Database, Azure SQL Data Warehouse, Azure Data Lake, Azure Cosmos DB, Azure Stream Analytics.
- Direct experience building data pipelines using Azure Data Factory and Apache Spark (preferably Databricks).
- Experience building data warehouse solutions using ETL/ELT tools such as SQL Server Integration Services (SSIS), Oracle Data Integrator (ODI), Talend, and Wherescape Red.
Additional Requirements:
- Experience with Azure Event Hub, IOT Hub, Apache Kafka, Nifi for use with streaming data/event-based data.
- Experience with other Open Source big data products eg Hadoop (incl. Hive, Pig, Impala)
- Experience with Open Source non-relational/NoSQL data repositories (incl. MongoDB, Cassandra, Neo4J)
- Experience working with structured and unstructured data including imaging & geospatial data.
- Comprehensive understanding of data management best practices including demonstrated experience with data profiling, sourcing, and cleansing routines utilizing typical data quality functions involving standardization, transformation, rationalization, linking, and matching.
- Experience working in a Dev/Ops environment with tools such as Microsoft Visual Studio Team Services, Chef, Puppet or Terraform.
Estimated Salary: £80,000 - £100,000 per annum.
Location: Flexible remote working options available. We are open to discussing alternative locations.
-
Belfast, United Kingdom Kainos Full timeDiscover a challenging opportunity as a Senior Data Engineer at Kainos, where you will design and develop data processing software components for solutions handling large datasets.We are seeking an experienced professional to join our agile team, providing strong development leadership and taking responsibility for significant technical components of data...
-
Data Engineer
5 days ago
Belfast, United Kingdom Data Intellect Full timeJob OverviewWe are seeking a highly skilled Data Engineer to join our team at Data Intellect. As a Data Engineer, you will be responsible for designing, developing, and implementing data-driven solutions that transform our clients' businesses.Key ResponsibilitiesDesign and Develop Data-Driven SolutionsCollaborate with Cross-Functional TeamsImplement and...
-
Large Scale Fit Out Manager
4 days ago
Belfast, United Kingdom HAYS Specialist Recruitment Full timeAbout the Job">As a seasoned Site Manager, you will lead our team on a cutting-edge large scale fit out project in Belfast City Centre. This pivotal role requires someone who can deliver exceptional results, maintain a safe working environment, and foster strong relationships with our team, senior management, and clients.">Your Key Responsibilities">">Lead...
-
Senior Data Platform Engineer
1 month ago
Belfast, United Kingdom Divvy Cloud Corp. Full timeAbout the RoleThe Data Platform Engineering team at Divvy Cloud Corp. is seeking a highly skilled Senior Data Platform Engineer to join our team. As a Senior Data Platform Engineer, you will be responsible for architecting and scaling streaming and batch data pipelines, while also designing the CI/CD infrastructure that ensures efficient development and...
-
Data Engineer
3 months ago
Belfast, United Kingdom Rapid7 Full timeData Engineer II Rapid7 seeks a highly motivated and inquisitive aspiring Data Engineer II to join our quickly scaling data engineering function. Come and join our efforts in unlocking the value of data through industry-leading innovation, cutting edge modern tooling, democratization at scale and building exceptional and trusted data products for the...
-
Data Engineer Position
4 weeks ago
Belfast, United Kingdom Rapid7 Full timeAbout the RoleThe Data Engineering team at Rapid7 is seeking a skilled Data Engineer to join our fast-paced and innovative environment. As a Data Engineer, you will be responsible for designing, building, and maintaining large-scale data pipelines and infrastructure to support our enterprise data.Key ResponsibilitiesBuild and maintain data pipelines using...
-
Data Engineer
4 weeks ago
Belfast, United Kingdom Data Intellect Full timeJob DescriptionWhat you’ll be doing:Build and enhance bespoke Data Science and Engineering solutions for our clientsEnhance and grow your data science and engineering knowledge among subject matter experts in our learning and development ecosystem Deliver high quality, best practice software solutions Use a range of architectures, tools and...
-
Civil Engineering Data Management Lead
4 days ago
Belfast, United Kingdom MCS Group Full time**Job Summary**MCS Group is seeking a highly skilled Civil Engineering Data Management Lead to join our team in Belfast. As a key member of our organization, you will be responsible for overseeing the management of data and civil engineering projects, ensuring seamless delivery from initiation to completion.**About the Role**The successful candidate will...
-
Lead Data Engineer
5 days ago
Belfast, United Kingdom Divvy Cloud Corp. Full timeJob SummaryWe are seeking a Lead Data Engineer to join our Data Platform Engineering team at Divvy Cloud Corp. As a key member of the team, you will be responsible for leading the design and implementation of scalable data pipelines using Apache Kafka, Apache Iceberg, and Apache Spark.You will work closely with cross-functional teams to deliver highly...
-
Senior Data Engineer
4 weeks ago
Belfast, United Kingdom Rapid7 Full timeAs a key member of the Rapid7 Data Platform Engineering team, you'll have the opportunity to design and implement a highly scalable Data Platform that supports Change Data Capture (CDC) using Debezium and Kafka for data replication across different databases and services. About the Role The Data Platform team is responsible for building ETL pipelines that...
-
Data Management Specialist
4 days ago
Belfast, United Kingdom MCS Group Full timeJob Title: Data Management SpecialistAre you a skilled professional with experience in data management and civil engineering? Do you have a passion for leading projects and working with multidisciplinary teams?We are seeking a highly motivated and organized Data Management Specialist to join our team at MCS Group. As a key member of our team, you will be...
-
Senior Data Engineering Specialist
4 weeks ago
Belfast, United Kingdom PA Consulting Full timeData Engineering Team LeadWe are seeking a highly skilled Data Engineering Team Lead to join our team at PA Consulting. As a Data Engineering Team Lead, you will be responsible for leading teams to design, build, and maintain large-scale data solutions and applications using AWS data and analytics services.Key Responsibilities:Lead teams to design, build,...
-
Data Scientist and Engineer
4 weeks ago
Belfast, United Kingdom Data Intellect Full timeJob DescriptionWe are seeking a highly skilled Data Scientist and Engineer to join our team at Data Intellect. As a key member of our data science and engineering team, you will be responsible for designing, developing, and implementing data-driven solutions for our clients.Key Responsibilities:Design and develop bespoke data science and engineering...
-
Senior Data Architect
4 days ago
Belfast, United Kingdom Divvy Cloud Corp. Full timeAbout the RoleThe Principal Engineer position is a key member of our Data Platform Engineering team at Divvy Cloud Corp. In this role, you will be responsible for designing and implementing scalable data pipelines using Apache Kafka, Apache Iceberg, and Apache Spark.You will work closely with cross-functional teams to deliver highly available, performant,...
-
Data Engineering Lead
1 month ago
Belfast, United Kingdom Corvus Full timeAbout the Role:This is a key position in the business, requiring a strong leader to guide the data engineering team. The successful candidate will have experience in managing teams, engaging with clients, and delivering high-quality solutions. The role involves working on various projects across different industries, including energy and defence. The ideal...
-
HR Administrator
5 months ago
Belfast, United Kingdom MPA Recruitment Full time**Job Role: HR Administrator **(scale 4) We are recruiting a **HR Administrator (scale 4 ) **for our public service client based in Belfast. To deliver a fire and rescue service and work in partnership with others to ensure the safety and well-being of our community. To provide a modern, efficient, professional and customer-focused administrative service...
-
Legal Data Administrator
5 months ago
Belfast, United Kingdom Abacus Professional Recruitment Full time**Are you interested in a fantastic new opportunity within a global law firm?** **Do you have experience with data entry?** **Have you got a keen eye for detail?** **The company** Our client is an international law firm who are operating a new function in Belfast City Centre and are seeking to recruit a Legal Data Administrator to join their growing...
-
Principal Software Engineer
2 months ago
Belfast, United Kingdom Rapid7 Full timeAs a Principal Engineer, you’ll get the opportunity to be a hands-on engineer, learning best practice engineering processes and approaches whilst receiving ongoing development through coaching, mentoring and pairing with other engineers on your team. From problem-solving to challenging old ways of thinking, you will have the opportunity to unleash your...
-
Data Engineer
1 month ago
Belfast, United Kingdom PA Consulting Full timeJob Title: Data Engineer - Cloud ArchitectPA Consulting is seeking a highly skilled Data Engineer - Cloud Architect to join our team. As a key member of our Digital team, you will be responsible for designing and building large-scale data solutions and applications using AWS data and analytics services.Key Responsibilities:Design and build ETL pipelines...
-
Data Platform Architect
7 days ago
Belfast, United Kingdom Rapid7 Full timeAbout the RoleAs a Principal Engineer on our Data Platform Engineering team, you will be responsible for architecting and scaling streaming and batch data pipelines. You will design the CI/CD infrastructure that ensures efficient development and deployment of data services. You will play a key role in shaping the architecture of our data platform,...