See more Collapse

Data Engineer

1 month ago

Camden Area, United Kingdom Vallum Associates Full time
RequirementsnAt least two years of professional experience as a data engineer with a proven ability to provide effective data solutions in a production setting.nKnowledge of developing real-time data stream systems (ideally Kafka).nProven track record in developing data systems using PySpark and Apache Spark for batch processing.nCapable of managing data intake from various sources, including data streams, unstructured data, relational databases, and NoSQL databases.nExtensive knowledge in distributed systems, cloud architecture, and data pipelines.nProficiency in Python programming (knowledge of Scala or Rust is a plus).nFamiliarity with ETL principles in contemporary data applications (Dagster, Airflow, Perfect).nFamiliarity with AWS services such as Glue, Redshift, Athena, and S3.nProficiency with Terraform, Kubernetes, and ArgoCD (expertise not required; cloud team support available).nThorough understanding of software engineering techniques, including version control, continuous integration and delivery (CI/CD), testing, and documentation.nWell-versed in data lifecycle management, including development, storage, and access.nAcquainted with Jira, Confluence, and other agile work and collaboration tools.nA cooperative team member with strong communication skills to interact with various internal and external teams and stakeholders.nCapable of operating in a fast-paced setting.nAdvocate of continuous development and open-mindedness.nProficiency in English, with the ability to work in the EU being a bonus.

What the Role EntailsnAs a growing company with expanding data demands, scalable, dependable, and efficient data architecture is required. We are now seeking a Data Engineer to join our Data team to help build, upgrade, and enhance our current data architecture.nDevelop and automate streaming data architecture, supporting the expansion of the data team in an interesting full-time position with plenty of challenges.nCollaborate closely with other data engineers, scientists, analytics teams, and cloud engineering teams to develop and manage data products across multiple domains, providing scalable and dependable data for internal decision-making and client visibility.nPlay a major role in the development and optimisation of data systems to support our Data Lake/Data Mesh architecture and data processing needs, ensuring the provision of scalable, high-performance data products for the entire organisation and proper data governance.nUtilise and improve our current AWS-based data platform.nWork with our tech stack, which includes dbt/DuckDB for transformation, Kafka/RabbitMQ as a streaming platform, Deltalake as a data format, Dagster for managing data assets, and Terraform, Kubernetes, and ArgoCD as the underlying infrastructure.

Additional ResponsibilitiesnBuild tools to produce data products and automate the self-serve platform.nCreate dependable and highly scalable data assets using Python, Scala, Spark/PySpark.nManage both fresh and current data sources into our pipelines for ELT and reverse ETL.nCreate a data strategy that uses appropriate resources to manage, classify, and locate important data while guaranteeing safe access.nUse CI/CD, testing, and standard monitoring procedures to guarantee data quality and dependability.nContinuously seek out and implement new technologies and industry best practices to enhance our data infrastructure.nWork within the tech team using an agile methodology.nUnderstand the business to comprehend the data demands of various internal and external stakeholders and design the best possible solutions.

This position offers the opportunity to work with a bright and varied team to develop and improve our data platform for data processing, access, gathering, and monitoring, building the future of our data infrastructure.

We have other current jobs related to this field that you can find below

  • Data Engineer

    4 weeks ago

    Camden Area, United Kingdom La Fosse Full time

    Data EngineerLocation:London, UKnHybrid:3x a week in officenSalary:£63,000About the ClientnOur client is a top-tier professional services firm known for its exceptional workplace culture and some of the most sought-after office spaces in London.Role OverviewnOur client is in search of a skilled Data Engineer to manage and engineer the data warehouse. The...

  • Data Engineer

    2 weeks ago

    Camden Area, Greater London, United Kingdom Scalexperts Full time

    Data Engineer - Databricks - £70-75,000 - London/HybridScalexperts are partnered with an education business who after making a few recent senior leadership hires in Data & Technology are now scaling their Data Engineering function.They are looking for a Data Engineer to join them to work on building a brand new data platform which is still in early stage...

  • Data Engineer

    2 weeks ago

    Camden Area, Greater London, United Kingdom Response Informatics Full time

    We are looking for a Lead Data Engineer to join our Data Engineering Team. In this senior role, you will be responsible for managing our Enterprise Data Warehouse and ensuring the efficient flow and quality of data.nYour tasks will include implementing data processes, managing data architecture, designing ETL processes, and analyzing data for business...

  • Data Engineer

    4 weeks ago

    Camden Area, United Kingdom Xcede Full time

    Data EngineernTech for good - up to £52,000n1 day a week - LondonMy Client is in the tech for good space working to protect children globally. They are looking for a Data Engineer who will report directly to the Head of Data and is responsible for building and maintaining of the Enterprise Data Platform, including data ingestion pipelines and data...

  • Data Engineer

    2 weeks ago

    Camden Area, Greater London, United Kingdom Hunter Bond Full time

    Job title:Data EngineerClient: Elite Prop Trading FirmSalary: Up to £1500 per dayLocation: London/HybridAre you a skilled Data Engineer looking for an exciting opportunity in a cutting-edge fintech company?Join our client's team and take ownership of designing and implementing data solutions. Your expertise in data extraction, transformation, and loading...

  • Data Engineer

    4 weeks ago

    Camden Area, United Kingdom Understanding Recruitment Full time

    Data EngineerDo you want to join aleadinginvestment firm specializing inquantitativetrading, driven by innovation and excellence? Keep reading!Location : London, UKAs aData Engineer , you'll build and maintain infrastructure for trading strategies, collaborating with data scientists and analysts.Responsibilities:nData PipelinesnData Integrationn️...

  • Data Engineer

    4 weeks ago

    Camden Area, United Kingdom Prism Digital Full time

    Lead Azure Data Engineer | PySpark (Python) & Synapse | Tech for Good / CharityEssential skills required:nAzure – solid experience required of the Azure Data ecosystemnPython - ESSENTIAL asPySparkis used heavily. You will be tested on PySparknAzureSynapse- ESSENTIAL as is used heavilynSparknAzure Data Lake / Data Bricks / Data FactorynBe happy to act as a...

  • Data Engineer

    2 weeks ago

    Camden Area, Greater London, United Kingdom Hexegic Full time

    Data Engineers are responsible for aiding UK Health in developing a cleaned, validated, and discoverable data foundation to enable new analytical workflows and applications.nThey will need to work closely with UK Health subject matter experts to scope new data integrations, design a data model, and perform data cleansing and transformation within the data...

  • Data Engineer

    1 month ago

    Camden Area, United Kingdom Travelex Full time

    Job Title: Data EngineernJob Type: Full Time, PermanentnWorking location: London, HybridRole PurposeAt Travelex we are developing modern data services, which will be at the heart of our relationship with our customers. Our data architecture is becoming more service-oriented and more real-time – which creates a range of engineering problems for you to...

  • Data Engineer

    4 weeks ago

    Camden Area, United Kingdom Understanding Recruitment Full time

    Fantastic new opportunity for a Data Engineer to join a high performing team at a leading hedge fund!You'll be vital in building and maintaining infrastructure for the efficient execution of their quantitative and discretionary trading strategies.What we'd love to see:MSc in Comp Sci, Engineering etc. from a Top UniversitynDemonstrable Data Engineering...

  • Data Engineer

    1 month ago

    Camden Area, United Kingdom Harrington Starr Full time

    Data EngineernOn-Premise / Excel / SQLWe have partnered with a leading client based in Central London who are looking to increase their data engineering team.Role:nYou will be working on a large volume of various financial reporting data sources. The client here is using on-premise databases including Excel services & SQL.nYou will be working to gather data...

  • Data Engineer

    2 weeks ago

    Camden Area, Greater London, United Kingdom Harrington Starr Full time

    Data EngineernOn-Premise / Excel / SQLWe have partnered with a leading client based in Central London who are looking to increase their data engineering team.Role:nYou will be working on a large volume of various financial reporting data sources. The client here is using on-premise databases including Excel services & SQL.nYou will be working to gather data...

  • Data Engineer

    2 weeks ago

    Camden Area, Greater London, United Kingdom Mondrian Alpha Full time

    Our client is a global investment management firm that utilizes long-short equities, long-only, fixed income and commodities futures strategies across its funds seek to add a experienced Data Engineer Risk Developer.They have deep expertise in trading, technology and operations and attribute their success to rigorous scientific research. As a technology and...

  • Data Engineer

    4 weeks ago

    Camden Area, United Kingdom Amicus Full time

    Data Engineer – London – Up to £120,000I am working with a leading Blockchain start-up, whose core offering is based around the research, development and deployment of infrastructure.This VC-backed client has seen massive success in the last 12 months and are now capitalizing on their extremely strong position to scale their engineering...

  • Data Engineer

    2 weeks ago

    Camden Area, Greater London, United Kingdom Amicus Full time

    Data Engineer – London – Up to £120,000I am working with a leading Blockchain start-up, whose core offering is based around the research, development and deployment of infrastructure.This VC-backed client has seen massive success in the last 12 months and are now capitalizing on their extremely strong position to scale their engineering...

  • Data Engineer

    4 weeks ago

    Camden Area, United Kingdom Eames Consulting Full time

    Eames Consulting are delighted to be supporting an established Investment management company that are looking for a Data engineer.Responsibilities:Develop and maintain complex SQL queries, customize reports, and optimize data usage.nCreate tools to address business challenges and empower users with third-party tools.nDesign and implement cloud-based data...

  • Data Engineer

    2 weeks ago

    Camden Area, Greater London, United Kingdom Eames Consulting Full time

    Eames Consulting are delighted to be supporting an established Investment management company that are looking for a Data engineer.Responsibilities:Develop and maintain complex SQL queries, customize reports, and optimize data usage.nCreate tools to address business challenges and empower users with third-party tools.nDesign and implement cloud-based data...

  • Data Engineer

    1 month ago

    Camden Area, United Kingdom Singular Recruitment Full time

    Data Engineer (one day a week in the central London office)Sports AnalyticsThis role is a unique opportunity to combine technical challenges with creativity in a collaborative, high-standard work environment. By joining this team, you’ll not only be part of a creative and open work culture focused on innovation and excellence but also have the chance to...

  • Data Engineer

    4 weeks ago

    Camden Area, United Kingdom Seargin UK & Ireland Full time

    Data Engineers needed for 6-12 month project based in London paying £450 a day inside IR35. This is an Urgent role an ideally we need the data engineers to be able to start asap or within 3 weeks as a maximum. All data engineers have to able to work in London 3 days a week, have a valid visa as we Are not able to sponsornTechnical Stack:- Python, Postgres...

  • Data Engineer

    2 weeks ago

    Camden Area, Greater London, United Kingdom Seargin UK & Ireland Full time

    Data Engineers needed for 6-12 month project based in London paying £450 a day inside IR35. This is an Urgent role an ideally we need the data engineers to be able to start asap or within 3 weeks as a maximum.All data engineers have to able to work in London 3 days a week, have a valid visa as we Are not able to sponsornTechnical Stack:Python, Postgres SQL,...