Gcp Data Engineer
1 week ago
We're looking for a capable Senior Data Engineer, who will be part of the Data team. Our client's mission is to build a robust platform and pipelines to collect and publish data from/to multiple sources, both internal and external.
Using a combination of batch and real-time streaming technologies, the role will facilitate cutting-edge analytics, real-time decisioning, and robust reporting. We believe in data being at the core of the business. We hate latency, approximation and baddata quality
This role is a great opportunity for people who love data, software architecture, streaming technologies, distributed systems and who want to have a real-world impact by building production systems with a world class team.
**Key Accountabilities**:
- Managing and administering our BigQuery Data Warehouse to guarantee continuous and high availability of the data.
- You will mainly be working with GCP (Cloud Storage, BigQuery, Composer, DataFlow), Confluent for Kafka and Snowplow.
- Working with the extended team to understand their needs and translating them into fantastic data products, this includes working with and alongside Machine Learning researchers and data scientists on cutting edge ML models.
- Designing and building pipelines to collect data from various data sources.
- Designing and building Dataflow streaming pipelines to process events against ML models and publish results.
- Supporting data modelling using DBT on top of GCP BigQuery.
- Working with backend, frontend, DevOps & QA engineers to ensure that data events are well-designed and correctly integrated to data pipeline services.
- Helping us implement a data-driven mindset in the company.
**Requirements**:
- At least 5+ years' experience as a data engineer especially using realtime & batch data in a production environment.
- Understanding of Data Warehousing and Data Lake principles (including indexing, query graphs, basic administration, relational models, etc).
- Experience using streaming technologies such as Kafka/PubSub/Kinesis.
- You have a proven track record with a programming language. We are especially keen on any or all of the following Python, Go and SQL, specifically in connection with AirFlow and DBT respectively.
- Experience designing and building production systems on GCP and/or AWS infrastructure.
- Experience using distributed systems at scale in a production environment using technologies such as Beam(GCP Dataflow)/Spark.
- Strong understanding of data quality principles.
- You are experienced with database technologies. (Best Practice, Performance Optimisation, Fault Finding).
- You are experienced working within a SDLC (inc. Testing, Git, Design Documentation, Agile Delivery - SCRUM
**Nice to Have**
- Experience with TerraForm/K8's.
- Have mentored/supported team members.
- Experience deploying and maintaining both batch and real-time machine learning models.
- Understand distributed systems and architecture for scale.
- Understand Security and InfoSec pain points in Data engineering.
- Fintech/Finance/Payments/Retail Banking experience
-
Gcp Data Engineer
7 days ago
London, United Kingdom SR2 Full timeGCP Data Engineer / Data Warehouse Engineer Data Engineer / Data Warehouse / BIG Query / ETL / SQL / GCP / Cloud / Pipelines / Inside IR35 Contract: Inside IR35 Length: 6 Months (rolling every additional 6 months) Rate: £650 per day Location: Fully Remote The Role SR2 are excited to be working with an industry leading energy provider, they have...
-
Gcp Data Engineer
5 days ago
London, United Kingdom Explore Group Full timeGCP - DATA ENGINEER - HYBRID - UP TO 70K I'm currently working with a leading Media company who are searching for a talented Hybrid Data Engineer! They'd require you to come to the office in London 2/3 times a week. The client i am working alongside is one of Europe's leading media and entertainment companies with millions of customers across multiple...
-
Gcp Head of Delivery
7 months ago
London, United Kingdom Data Idols Full time**Head of Delivery**: **Salary: £90,000 - £100,000**: **Location: Remote**: Data Idols is working with a Google Cloud consultancy based in the UK with clients across the UK, Europe and America. We are looking for a Head of Delivery to help this fast-growing consultancy deliver specialist GCP-specific projects for their many and varied clients. As a Head of...
-
Data Engineer
6 days ago
London, United Kingdom Data Idols Full time**Data Engineer** Would you like to build enterprise scale data solutions for a global organisation that reaches 400 million users monthly? We are supporting an ambitious data-driven client in the world of Multimedia with their search for a Data Engineer to develop and maintainnew features of their data platform and end-to-end delivery of exciting data...
-
Gcp Engineer
3 days ago
London, United Kingdom Harnham Full time**GCP ENGINEER,** **6-MONTH CONTRACT** **LONDON (FULLY REMOTE)** **£450-£550 PER DAY** This position as a GCP Engineer allows you to work within a dynamic consultancy company located in the heart of London. If you are tired of working repetitive projects that tend to merge into BAU, this contract will allow to work on technically varied projectsdaily....
-
Gcp Data Engineer
21 hours ago
London, United Kingdom Method Resourcing Full time**Data Engineer | Python | SQL | GCP | BigQuery | Data Studio | ETL | DBT | Stitch | Fivetran | Singer | £40,000-£55,000** Method Resourcing are currently working with a Media Publication company based in London that report on the world of data and provide publications to over 300,000 data professionals all over the world! After receiving an injection of...
-
Data Engineer
2 months ago
London, United Kingdom CV-Library Full timeData Engineer / GCP / Digital £500 - £550 (Outside IR35) London - Hybrid 6 Month Contract My client is a global ecommerce retailer based in London who are looking for a Data Engineer to join their dynamic data team to design, build and maintain their data pipelines in a cloud native environment. You'll be working with the team on a data...
-
Data Engineer
2 months ago
London, United Kingdom CV-Library Full timeData Engineer / GCP / Digital £500 - £550 (Outside IR35) London - Hybrid Ideally January Start - 6 Month Contract My client is a global ecommerce retailer based in London who are looking for a Data Engineer to join their dynamic data team to design, build and maintain their data pipelines in a cloud native environment. You'll be working...
-
Gcp Data Engineering Lead
1 week ago
London, United Kingdom X4 Group Full timeA client within the financial sector are seeking a GCP Data Engineering Lead to join their IT team on a 6 month contract basis with. The role is deliverable on a fully remote basis. The client requires a GCP Data Lead with significant experience in leadership position. The role will involve work on multiple upcoming projects whilst ensuring all relevant...
-
GCP Data Architect
1 day ago
London, United Kingdom Pioneer Search Full timeGCP Data Architect: GCP, BigQuery, Data Lake, Pub/SubA leading Tech Consultancy is seeking a GCP Data Architect for an upcoming large-scale migration to GCP. As the Data Consultant, you will play an integral role in mapping out and defining the GCP Data Architecture for a large-scale digitisation programme, including the design and development of a GCP Data...
-
Data Engineer
4 weeks ago
London, United Kingdom CV-Library Full timeData Engineer / GCP / Digital£500 - £550 (Outside IR35)London - HybridIdeally January Start - 6 Month ContractMy client is a global ecommerce retailer based in London who are looking for a Data Engineer to join their dynamic data team to design, build and maintain their data pipelines in a cloud native environment. You'll be working with the team on a data...
-
GCP Data Engineering Lead
2 days ago
London, Greater London, United Kingdom Pioneer Search Full timeJob Overview:We are seeking a GCP Data Engineering Lead to join our team in designing and implementing scalable data architecture solutions for a large-scale digitisation programme. This is an exciting opportunity to work with cutting-edge technologies and make a significant impact on the company's digital transformation journey.About the Role:The GCP Data...
-
Data Engineer GCP Professional
3 weeks ago
London, Greater London, United Kingdom TRIA Full timeJob Description:TRIA, a global ecommerce retailer, is seeking a skilled Data Engineer to join their dynamic data team. Based in London, this role involves designing, building, and maintaining cloud-native data pipelines in Google Cloud Platform (GCP).The ideal candidate will have extensive experience in GCP, specifically in building data pipelines, and...
-
GCP Data Architect
6 hours ago
City Of London, United Kingdom Pioneer Search Full timeGCP Data Architect: GCP, BigQuery, Data Lake, Pub/SubA leading Tech Consultancy is seeking a GCP Data Architect for an upcoming large-scale migration to GCP. As the Data Consultant, you will play an integral role in mapping out and defining the GCP Data Architecture for a large-scale digitisation programme, including the design and development of a GCP Data...
-
Lead Data Engineer
5 days ago
London, United Kingdom Data Idols Full time**Lead Data Engineer (GCP)** Would you like to build enterprise scale data solutions within a GCP environment for a global multimedia organisation that reaches 400 million users monthly? Through a mix of technical leadership and hands on delivery, as a Lead Data Engineer, you will beinvolved in delivering solutions from initial concept to production. **The...
-
Senior Engineer, Data Engineering
2 weeks ago
London, United Kingdom Data Idols Full time €75,000 - €100,000Senior Data Engineer Joining one of the best-known brands in the UK as a Senior Data Engineer, you will support the delivery of data transformation during an exciting time within the company. They are looking for someone with experience in building and maintaining data pipelines using modern data technologies in a cloud environment. For this role as...
-
Senior Engineer, Data Engineering
2 weeks ago
London, United Kingdom Data Idols Full time €75,000 - €100,000Senior Data Engineer Joining one of the best-known brands in the UK as a Senior Data Engineer, you will support the delivery of data transformation during an exciting time within the company. They are looking for someone with experience in building and maintaining data pipelines using modern data technologies in a cloud environment. For this role as...
-
Data Architect Gcp
5 days ago
London, United Kingdom Opus Recruitment Solutions Ltd Full time**GCP Data Architect - Remote - 3 month contract - £500.00 to £550.00 per day (Outside IR35)** I am currently recruiting for a GCP Data Architect to join a global consultancy client of ours. This position has been deemed outside IR35 and can be completed on a remote basis (though must be based in the UK). **Key Skills**: - Hands on GCP Data...
-
Cloud Data Architect
2 weeks ago
London, Greater London, United Kingdom Insight International (UK) Ltd Full time**Job Title:** Cloud Data Architect - GCPAbout Us: At Pracyva, we're pushing the boundaries of data engineering and architecture. As a leading provider of innovative solutions, we're seeking a highly skilled Cloud Data Architect - GCP to join our team.Salary Estimate: $180,000 - $220,000 per yearJob DescriptionDesign, deploy, and manage Databricks clusters...
-
GCP Data Platform Expert
3 weeks ago
London, Greater London, United Kingdom Wave Talent Full timeData Engineering OpportunityWe are looking for a highly skilled GCP Data Platform Expert to lead the migration of our client's data platforms and warehousing to Google Cloud Platform.About the Role:Leverage your expertise in designing and implementing scalable data platforms on GCP.Work closely with our team to develop and maintain high-quality data...