Data Engineer – Talent Pipeline
4 weeks ago
Job Description
The Data Engineer will be implementing data ingestion and transformation pipelines for large scale organisations. We are seeking someone with deep technical skills in a variety of technologies to play an important role in developing and delivering early proofs of concept and production implementation.
You will be building solutions using a variety of open-source tools & Microsoft Azure services, and a proven track record in delivering high-quality work to tight deadlines.
Your main responsibilities will be:
Designing and implementing highly performant data ingestion & transformation pipelines from multiple sources using a variety of technologies Delivering and presenting proofs of concept of key technology components to prospective customers and project stakeholders. Developing scalable and re-usable frameworks for ingestion and transformation of large data sets Master data management system and process design and implementation. Data quality system and process design and implementation. Integrating the end to end data pipeline to take data from source systems to target data repositories ensures the quality and consistency of data is maintained at all times Working with event-based / streaming technologies to ingest and process data Working with other members of the project team to support the delivery of additional project components (Reporting tools, API interfaces, Search) Evaluating the performance and applicability of multiple tools against customer requirements Working within an Agile delivery / DevOps methodology to deliver proof of concept and production implementation in iterative sprints.
Qualifications
Hands-on experience designing and delivering solutions using the Azure Data Analytics platform (Cortana Intelligence Platform) including Azure Storage, Azure SQL Database, Azure SQL Data Warehouse, Azure Data Lake, Azure Cosmos DB, Azure Stream Analytics Direct experience in building data pipelines using Azure Data Factory and Apache Spark (preferably Databricks). Experience building data warehouse solutions using ETL / ELT tools such as SQL Server Integration Services (SSIS), Oracle Data Integrator (ODI), Talend, and Wherescape Red. Experience with Azure Event Hub, IOT Hub, Apache Kafka, Nifi for use with streaming data / event-based data Experience with other Open Source big data products eg Hadoop (incl. Hive, Pig, Impala) Experience with Open Source non-relational / NoSQL data repositories (incl. MongoDB, Cassandra, Neo4J) Experience working with structured and unstructured data including imaging & geospatial data. Comprehensive understanding of data management best practices including demonstrated experience with data profiling, sourcing, and cleansing routines utilizing typical data quality functions involving standardization, transformation, rationalization, linking and matching. Experience working in a Dev/Ops environment with tools such as Microsoft Visual Studio Team Services, Chef, Puppet or Terraform-
Senior Data Engineer
6 days ago
Birmingham, United Kingdom Clearwater People Solutions Ltd Full timeOur client, a well-known and market-leading Financial Services company, is currently recruiting for a Senior Data Engineer to join their Data engineering team. The Senior Data Engineer will be responsible for designing, developing, and maintaining data pipelines and workflows to support the organisation's data processing needs and facilitate BI reporting and...
-
Data Engineer New
1 month ago
Birmingham, Birmingham, United Kingdom Experis UK Full timeData EngineerRole purpose:The Data Engineer position is an integral part of the Data team within the Digital and Information function. This role entails overseeing the availability, integrity, quality, and accuracy of data. The primary responsibilities include designing, developing, and maintaining data pipelines and workflows to support the organisation's...
-
Careers Pipelines Co-ordinator
6 days ago
Birmingham, United Kingdom SSE Full time**About the Role** **Base Location**: Birmingham / Coventry - Flexible | We're proud to offer a Flexible First approach to work, and this role is no exception. We will, however need your base 'home' location to be one of our SSE offices or depots around the UK and Ireland. **Salary**: £32,900 - £48,000 + performance-related bonus + a range of other...
-
Senior Engineer, Data Engineering
4 weeks ago
Birmingham, United Kingdom Experis - ManpowerGroup Full timeSenior Data Engineer Role purpose: The Senior Data Engineer plays a pivotal role within the Data team of the Digital and Information function, assuming a leadership position in ensuring the availability, integrity, quality, and accuracy of data. With a wealth of experience, the Senior Data Engineer leads the design, development, and maintenance of data...
-
Senior Data Engineer
1 month ago
Birmingham, United Kingdom VIQU Limited Full time**Senior Data Engineer - £45-86k - UK Remote Working - Must have SC Clearance** Our client, a leading consultancy business is looking for outstanding talent to join their vibrant and growing team.With cutting-edge resources, are you the next hire to join their passionate team. **Main responsibilities of the Senior Data Engineer**: As a Senior Data...
-
Senior Data Engineer
1 month ago
Birmingham, United Kingdom Undisclosed Company Full timeSenior Data EngineerRole purpose:The Senior Data Engineer plays a pivotal role within the Data team of the Digital and Information function, assuming a leadership position in ensuring the availability, integrity, quality, and accuracy of data. With a wealth of experience, the Senior Data Engineer leads the design, development, and maintenance of data...
-
Senior Data Engineer
1 month ago
Birmingham, Birmingham, United Kingdom Undisclosed Company Full timeSenior Data EngineerRole purpose:The Senior Data Engineer plays a pivotal role within the Data team of the Digital and Information function, assuming a leadership position in ensuring the availability, integrity, quality, and accuracy of data. With a wealth of experience, the Senior Data Engineer leads the design, development, and maintenance of data...
-
Senior Data Engineer
4 weeks ago
Birmingham, United Kingdom Experis - ManpowerGroup Full timeSenior Data Engineer Role purpose: The Senior Data Engineer plays a pivotal role within the Data team of the Digital and Information function, assuming a leadership position in ensuring the availability, integrity, quality, and accuracy of data. With a wealth of experience, the Senior Data Engineer leads the design, development, and maintenance of data...
-
Head of Data Engineering
4 weeks ago
Birmingham, United Kingdom LRQA Full timeWho are LRQA? LRQA stands for dedication to clients, market firsts, and deep expertise in risk management. We’ve grown to become a leading global assurance provider, bringing together outstanding expertise in certification, customised assurance, cybersecurity, inspection and training. While we’re proud of our heritage, it’s who we are today that...
-
Pipeline Consultant
3 weeks ago
Birmingham, United Kingdom CMR & Life Sciences Full timeKey Responsibilities: Utilise extensive experience in Transmission Pressure (High Pressure) Natural gas projects in the UK and Overseas Apply expertise in Distribution Pressure (HP, MP & LP) natural gas projects in the UK Contribute to onshore pipeline engineering projects in the UK, including design, construction, and contracting Demonstrate...
-
Senior Data Engineer
3 weeks ago
Birmingham, United Kingdom Xcede Full timeSENIOR DATA ENGINEER(EU Remote Opportunity)Xcede are delighted to be working with a very exciting Web3 / Blockchain organisation with a remote working culture. Their company makes life easier for organisations in the Web3 space by providing managed services across a range of operational tasks and responsibilities in one space.As a Senior Data Engineer in the...
-
Junior Talent Coordinator
1 month ago
Birmingham, United Kingdom Sparta Global Full time_(Junior Talent Acquisition/Recruitment coordinator)_ **Department**: Talent Team **Reporting to**:Senior Talent Lead / Head of Talent **Location**: Birmingham **Salary**: £20,000 basic per annum (Uncapped OTE £5670 per annum) **WE ARE SPARTA GLOBAL** Sparta Global is a leading provider of technology consulting services providing cross-functional...
-
Senior Engineer, Data Engineering
2 days ago
Birmingham, United Kingdom Lorien Full timeJob Description Data Engineer ~ Hybrid, West- Midlands ~£56k + bonus & great benefits Our client are a fast growing SaaS scale-up, revolutionising supply chain ecosystems and increasing revenue for their customers. You'll contribute to the organisation's high-quality data engineering practices, participating in technical design, implementation,...
-
Senior Engineer, Data Engineering
3 weeks ago
Birmingham, United Kingdom Lorien Full timeData Engineer ~ Hybrid, West- Midlands ~£56k + bonus & great benefits Our client are a fast growing SaaS scale-up, revolutionising supply chain ecosystems and increasing revenue for their customers. You'll contribute to the organisation's high-quality data engineering practices, participating in technical design, implementation, testing,...
-
Senior Engineer, Data Engineering
3 weeks ago
Birmingham, United Kingdom Lorien Full timeData Engineer ~ Hybrid, West- Midlands ~£56k + bonus & great benefits Our client are a fast growing SaaS scale-up, revolutionising supply chain ecosystems and increasing revenue for their customers. You'll contribute to the organisation's high-quality data engineering practices, participating in technical design, implementation, testing,...
-
Group Talent Acquisition Advisor
1 month ago
Birmingham, United Kingdom Richter Full time**About The Role**: The Group Talent Attraction Advisor is responsible for partnering with the business to provide 360 full-cycle recruitment. You'll provide excellence in the end-to-end resourcing and recruitment process - from developing attraction campaigns, through effective talent pipelining and succession planning, to sourcing, selection, offer...
-
Structural Engineer
1 month ago
Birmingham, United Kingdom Access Talent Group Full time((BST: 8:00 - 18:00 (Mon/Thurs) 8:00 - 16:00 (Fri))) Share this Job Access Talent Group are seeking a skilled Structural Engineer to join our Building Design team in Birmingham. The ideal candidate will have a strong background in structural engineering and a passion for creating innovative and sustainable building designs. As a Structural Engineer, you...
-
Talent Acquisition Lead
4 weeks ago
Birmingham, United Kingdom Summit HR Full timeI'm pipelining for a fantastic Talent Acquisition Lead role with an exciting business, you will lead a small team to ensure best practice and aim to deliver a first-class recruitment delivery service. This is global business that has lots of exciting change happening, TA here is fantastic place to be! The role involves: - Mentoring, nurturing a team of TA...
-
Machine Learning Engineer
3 weeks ago
Birmingham, United Kingdom Xpertise Recruitment Full timeXpertise is seeking two talented Machine Learning Engineers to join our esteemed team in Birmingham. As part of our growing engineering division, you will play a pivotal role in designing, implementing, and optimizing machine learning models and data pipelines. With a strong emphasis on AWS technologies and MLOps practices, you'll have the opportunity to...
-
Machine Learning Engineer
4 weeks ago
Birmingham, United Kingdom Xpertise Recruitment Full timeXpertise is seeking two talented Machine Learning Engineers to join our esteemed team in Birmingham. As part of our growing engineering division, you will play a pivotal role in designing, implementing, and optimizing machine learning models and data pipelines. With a strong emphasis on AWS technologies and MLOps practices, you'll have the opportunity to...
