Senior Data Integration Engineer

4 weeks ago

London, UK, United Kingdom Trafigura Full time
Main Purpose:
As a highly experienced Data Engineer in our Digital Transformation Team, you will play a crucial role in developing advanced data abstraction layers to facilitate seamless integration between legacy trading systems (built on WPF, .Net (C#, VB), Oracle, Java) and our modern AWS-based data and analytics stack.
Your expertise will drive the successful transformation of our data infrastructure, furthering the use of advanced analytics and data-driven decision-making across the company. You will manage a specialist team of engineers and scientists in order to achieve your objectives.

Knowledge Skills and Abilities, Key Responsibilities:

Abilities, Experience and Qualifications
  • Bachelor's or Master's degree in Software Engineering, Computer Science or a related field.
  • Proven experience in enterprise scale data engineering.
  • Expertise in working with legacy systems (SQL Server, Oracle, Java) and integrating them with cloud platforms, preferably AWS (S3, RDS, Lambda, Glue, Redshift).
  • Must be capable of working with multiple data structures, including but not limited to JSON, XML.
  • Exposure to data mapping/design tools such as Altova suite (MapForce), Schematron, ERWIN.
  • Worked with Apache Spark, Kafka, Hive, Hbase, Flume, Sqoop, Trino.
  • Proven experience with Object Relational Mapping systems (ORMs) and SOAP / REST, APIs.
  • Expert programming skills in Java and Python.
  • Familiarity with SQL and NoSQL databases, Data Modelling, and ETL processes, Change Data Capture (CDC).
  • Knowledge of data warehousing principles and architectures.
  • Experience with data governance and security.
  • Experience in a Commodity Trading, Financial Markets or Logistics Management environment is desirable.
Key Responsibilities
  • Develop abstraction layers to integrate disparate systems, ensuring data consistency, validity, traceability and accessibility for self-service by data scientists and analysts.
  • Work closely with stakeholders to understand data requirements and translate business needs into well-structured technical specifications.
  • Collaborate closely with our Trading Technology, Data Science and Cloud DevOps teams.
  • Manage a small team of engineers / computer scientists / data scientist
Competencies
  • Ability to solve difficult technical problems under pressure.
  • Ability to communicate effectively with a diverse set of stakeholders across business lines and technologies.
  • Understanding and experience implementing software engineering best practices.
  • Ability to manage and prioritise your and other's work.
Key Relationships and Department Overview:
  • You will report to the Head of the SSI Team & CTO who reports to the CIO and Chief Data Scientist.
  • The Digital Transformation Team is a joint venture between Trading Technology and the Data Science and Engineering Team.
Equal Opportunity Employer

We are an Equal Opportunity Employer and take pride in a diverse workforce. We do not discriminate in recruitment, hiring, training, promotion or other employment practices for reasons of race, colour, religion, gender, sexual orientation, national origin, age, marital or veteran status, medical condition or handicap, disability, or any other legally protected status.

  • Senior Data Engineer

    2 months ago

    London,, UK, United Kingdom Innova Recruitment Full time

    Senior Data EngineerESG Data BusinessLondon (Hybrid)£110,000 + benefitsDescriptionAn international ESG business are looking for a Senior Data Engineer to join their Data Team. With offices in the US and London as a business they support their clients in building a more sustainable future. With a large data repository, they can offer to business innovative...

  • Senior Data Engineer

    2 weeks ago

    London,, UK, United Kingdom Oliver Bernard Full time

    Senior Data Engineer Central London, 4 days a week onsite Salary paying £80k - £110k Opportunity to join a leading global alternative investments technology solutions partner within the wealth management industry. With headquarters in London and a global presence across 9 different locations, this is a chance to be part of a dynamic team driving innovation...

  • Senior Data Engineer

    1 week ago

    London,, UK, United Kingdom numi Full time

    Join a Deep Tech AI company with FAANG Leaders - Senior Data Engineer - Up to £100,000 Welcome to a world where data drives enterprise transformation with precision and innovation.Are you ready to join a cutting-edge company that's revolutionizing decision-making with its advanced AI platform?With their AI-driven solutions, they empower businesses to...

  • Senior Data Engineer

    2 weeks ago

    London,, UK, United Kingdom Ozone Talent Full time

    Senior Data Engineer Tech - Python, Spark, Terraform, AWS/ LambdaSalary – Up to £100,000Location – London/ HybridA rapidly expanding climate tech start-up is looking for a highly skilled Senior Data Engineer to join a growing and experienced team. You will focus on customer-facing systems as part of their platform team helping to desing and improve...

  • Senior Data Engineer

    2 weeks ago

    London,, UK, United Kingdom Cognify Search Full time

    Join a Tech-Scale up for a Global subscription business A Senior Data Engineer for a Global subscription business who have several exciting data projects underway. This is a technical data engineering role covering data platform, architecture and design, Ingestion and ML-Ops. The initial focus will be on building a near real time streaming ingestion...

  • Senior Data Engineer

    2 weeks ago

    London,, UK, United Kingdom Bauer Media Group Full time

    Senior Data EngineerBauer Media Audio London - Hybrid Sadly we are not able to sponsor candidatesBauer Media Audio is leading the charge in transforming our digital business, scaling innovative products and services across 9 dynamic territories. We're excited to announce the formation of our Centre of Excellence (CoE) within the Audio Data Organisation,...

  • Senior Data Engineer

    1 week ago

    London,, UK, United Kingdom Bauer Media Group Full time

    Senior Data EngineerBauer Media Audio London - Hybrid Sadly we are not able to sponsor candidatesBauer Media Audio is leading the charge in transforming our digital business, scaling innovative products and services across 9 dynamic territories. We're excited to announce the formation of our Centre of Excellence (CoE) within the Audio Data Organisation,...

  • Senior Data Engineer

    1 week ago

    London,, UK, United Kingdom Understanding Recruitment Full time

    Senior Data Engineer | IoT | Flex RemoteUp to £85,000 per annum (DOE) | Mostly remote, with meetups in London (monthly or quarterly)I am looking for a Senior Data Engineer to join a cutting edge start-up that’s leading the charge in IoT, and data-driven services across Europe! They are revolutionising the driving experience with cutting-edge in-car...

  • Senior Data Engineer

    1 week ago

    London, UK, United Kingdom Arcus Search Full time

    Arcus Search are working with a fast-growing start-up in the UK and they are looking for a Senior Data Engineer to join the team. I wondered if you'd be interested? Role: Senior Data Engineer Salary: £90,000 + 20% bonus Benefits: Medical, Life, Pension, L&D + more Location: Central London - 3 days a week in the office What we're looking for: 6+...

  • Senior Data Engineer

    3 months ago

    London,, UK, United Kingdom Xcede Full time

    SENIOR DATA ENGINEER(Hertfordshire HQ – circa x1 day a week in office. Commutable from London.)Xcede are delighted to be working with a well-renowned insurance company based in Hertfordshire. The company have an excellent Data Science and Machine Learning unit supported by a group of Data Engineers who are responsible for Data Pipelines. As a bonus - Data...

  • Senior Data Engineer

    4 weeks ago

    London,, UK, United Kingdom Peaple Talent Full time

    Senior Data Engineer | Fully Remote | up to £60,000Peaple Talent have partnered with a leading provider of land and property search information, who provide a range of services across digital mapping, environmental risk reports and sophisticated property management tools. Due to exciting growth plans in the Data team we are now seeking a selection of Senior...

  • Senior Data Engineer

    3 weeks ago

    London,, UK, United Kingdom Peaple Talent Full time

    Senior Data Engineer | Fully Remote | up to £60,000Peaple Talent have partnered with a leading provider of land and property search information, who provide a range of services across digital mapping, environmental risk reports and sophisticated property management tools.Due to exciting growth plans in the Data team we are now seeking a selection of Senior...

  • Senior Data Engineer

    2 weeks ago

    London,, UK, United Kingdom Arcus Search Full time

    Senior Data Engineer AWS - databricks - SQL - Python - TableauDo you want to be part of a dynamic and fast-growing organization, who deeply committed to advancing towards a global circular economy by addressing sustainability from the perspectives of people, the planet, and finance. We're looking for a talented Senior Data Engineer to join a new data...

  • Senior Data Engineer

    1 week ago

    London, UK, United Kingdom Arcus Search Full time

    Senior Data Engineer AWS - databricks - SQL - Python Do you want to be part of a dynamic and fast-growing organization, who deeply committed to advancing towards a global circular economy by addressing sustainability from the perspectives of people, the planet, and finance. We're looking for a talented Senior Data Engineer to join a new data team....

  • Senior Data Engineer

    2 weeks ago

    London,, UK, United Kingdom Arcus Search Full time

    Senior Data EngineerAWS - Databricks - SQL - Python - VisualizationDo you want to be part of a dynamic and fast-growing organization, who deeply committed to advancing towards a global circular economy by addressing sustainability from the perspectives of people, the planet, and finance.We're looking for a talented Senior Data Engineer to join a new data...

  • Senior Data Engineer

    2 days ago

    London,, UK, United Kingdom Arcus Search Full time

    Senior Data EngineerAWS - Databricks - SQL - Python - VisualizationDo you want to be part of a dynamic and fast-growing organization, who are deeply committed to advancing towards a global circular economy by addressing sustainability from the perspectives of people, the planet, and finance.We're looking for a talented Senior Data Engineer to join a new...

  • Senior Information Technology Engineer

    1 week ago

    London,, UK, United Kingdom Syntax Integration Limited Full time

    The CompanySyntax Integration is a boutique Managed Services Provider, primarily serving clients in the financial, property, and investment sectors. The RoleThis senior role is focused on the provision of hands-on technical leadership for the Service Desk. The Senior IT Engineer will ensure the provision of deep technical escalation capability, mentoring of...

  • Senior Data Engineer

    1 week ago

    London,, UK, United Kingdom Understanding Recruitment Full time

    Senior Data Engineer | Connected Car Services | Flex Remote Mostly remote (ideally once per month, but quarterly meet-ups acceptable).Up to £85,000 per annum DOEAre you passionate about transforming data infrastructure and driving innovation in the automotive industry?A leading connected car service and data-driven team are searching for a Senior Data...

  • Senior Data Engineer

    2 weeks ago

    London,, UK, United Kingdom Undisclosed Full time

    Senior Data Engineer £576 per day Umbrella London – Hybrid (2 / 3 days per week) 6 Month Contract Our client is currently searching for a Senior Data engineer to support with their team in London. Responsibilities • Execute migration of raw and derived datasets between on-prem and cloud data locations (e.g. GCP, Azure, AWS). Datasets magnitude vary...

  • Senior/Lead Data Engineer

    2 weeks ago

    London,, UK, United Kingdom Eden Smith Group Full time

    Senior/Lead Data EngineerLondon (1-2 days per week)/Remote (3-4 days per week)Salary up to £80,000 per annumAn exciting opportunity has arisen to join us here at Eden Smith in a permanent position. Due to expansion, the new Senior Data Engineer will be crucial in building robust data pipelines and data flows for our clients on Azure cloud platforms. They...