Data Engineer

4 weeks ago


South East London, United Kingdom Canada Life Full time
Location: London, Potters Bar, Bristol or Isle of Man (Hybrid working options available) Job Summary The Data Engineer is a technical role responsible for designing, developing, and maintaining data pipelines within the IT department. The pipelines will be realised in a modern lake environment and the engineer will collaborate in cross-functional teams

to gather requirements and develop the conceptual data models. This role plays a crucial part in driving data-driven decision-making across the organisation, ensuring data availability, quality, and accessibility for various business needs. What you'll do Data Pipeline Development Design, model, develop and maintain data pipelines to ingest, store, process, and present data. Ensure data quality, accuracy, and consistency. Collaborate with data architects to ensure data pipelines align with the overall data architecture strategy. Data Transformation and Integration Perform data transformation tasks, including data cleansing, enrichment, and aggregation, to prepare data for analytics and reporting. Integrate data from structured and unstructured sources, ensuring compatibility and alignment with data models and business requirements. Automate data transformation processes to improve efficiency. Data Quality Assurance Implement and maintain data quality checks and validation processes to identify and resolve data anomalies and errors. Monitor data pipelines for data quality issues and implement data quality improvements. Collaborate with business stakeholders to define data quality requirements. Data Modelling and Schema Design Collaborate with data architects and data scientists to design and implement data models, schemas, and structures. Ensure that data models support business reporting and analytics needs while optimizing query performance. Maintain data dictionaries and metadata to document data structures and relationships. Performance Optimization Optimize data storage, retrieval, and query performance by implementing indexing, partitioning, and caching strategies. Monitor data processing performance and address bottlenecks as they arise. Stay updated with best practices in data processing performance tuning. Documentation and Knowledge Sharing Create and maintain documentation for data pipelines, data transformation processes, and data integration procedures. Foster a culture of knowledge sharing within the data engineering team and across the organization. Collaboration and Stakeholder Engagement Collaborate effectively with cross-functional teams, data stakeholders, and business units to understand data requirements and deliver data solutions that meet business needs. Communicate technical concepts and data solutions to non-technical stakeholders in a clear and understandable manner. Must Have Skills/Experience Essential: Commercial experience of DataBricks, SQL, Python, Power BI Experience in data engineering, including designing and developing data pipelines for retrieval / ingestion / presentation / semantics in an Azure environment. Effective communication and collaboration skills to work with cross-functional teams and gather data requirements. Skills in data modelling (both structured and unstructured data) Desirable: Power Automate Data optimisation for performance, scalability, and efficiency. Experience in a financial corporation



  • London, United Kingdom Data Idols Full time

    Senior Data Engineer Location: London (Hybrid) Data Idols are working with one of the fast-growing companies in the logistics sector that are making the world a greener place. They are looking for a Senior Data Engineer to join their team within the Data Space. As a Senior Data Engineer, you will work alongside different data teams to design,...


  • London, United Kingdom Data Idols Full time

    Senior Data Engineer Location: London (Hybrid) Data Idols are working with one of the fast-growing companies in the logistics sector that are making the world a greener place. They are looking for a Senior Data Engineer to join their team within the Data Space. As a Senior Data Engineer, you will work alongside different data teams to design,...

  • Gcp Data Engineer

    4 weeks ago


    London, United Kingdom Data Idols Full time

    **GCP Data Engineer** Would you like to build enterprise scale data solutions for a global organisation that reaches 400 million users monthly? As a GCP Data Engineer, you will be involved in delivering solutions from initial concept to production. **The Opportunity** Working for a well known brand you will be responsible for all aspects of the data...


  • London, United Kingdom Abs Data Full time

    Lead Data Engineer Do you enjoy creating valuable insights with data? Do you want to play a key role in transforming our firm into an agile organization? Being agile will make us more responsive, more adaptable, and ultimately more innovative. We’re looking for a Data Engineer to: ~ transform data into valuable insights that inform business...


  • London, United Kingdom Abs Data Full time

    Regulatory Reporting, Big Data Software Engineer We’re looking for software engineers to: provide technology solutions that will solve business problems and strengthen our position as digital leaders in financial services work with our Regulatory Reporting team to translate business and external regulatory needs using Behavior Driven Development...


  • London, United Kingdom ON Data Staffing Full time

    We are seeking a dynamic and experienced Databricks professional to join our client, a growing startup consultancy, as a Lead Data Engineer. As Lead Data Engineer, you will play a pivotal role in driving the success of our client's Databricks practice through your expertise in presales, strong technical acumen, consulting skills, and effective team...

  • Data Engineer

    4 weeks ago


    London, United Kingdom Open Data Science Conferenc Full time

    You have prior experience as a data engineer, software engineer, or data architect. You are experienced in designing and production of solutions for complex problems. You thrive in a fast-paced, start-up environment. You seek to continuously improve processes and find the most optimal way to solve problems. You can work remotely and independently,...

  • Data Engineer

    4 weeks ago


    London, United Kingdom Abs Data Full time

    Data Engineer/Analyst Risk & Margin We’re looking for a software engineer to: provide technology solutions that will solve business problems in the Risk and Margin space work with risk managers to translate business needs using Behavior Driven Development (BDD) into functional requirements. Key to success will be the ability to understand the data...


  • London, United Kingdom Open Data Science Conferenc Full time

    You have prior experience as a data engineer, software engineer, or data architect. You can work remotely and independently, taking ownership of your work. You have proven English language communication skills that enable you to collaborate in a remote-first company with a diverse international workforce. Experience with Java11+ for ingesting,...

  • Java Data Engineer

    4 weeks ago


    London, United Kingdom Harnham - Data & Analytics Recruitment Full time

    **DATA ENGINEER £70,000 - £80,000 + BONUS REMOTE** Are you a Data Engineer with a strong background in Data Engineering looking for an exciting challenge? Want to become part of an exciting start-up in the telecoms industry as a Data Engineer? Read on! **THE COMPANY**: Harnham has partnered with an exciting Start-Up who thanks to their success, are...

  • Lead Data Engineer

    4 weeks ago


    London, United Kingdom Harnham - Data & Analytics Recruitment Full time

    **LEAD DATA ENGINEER** **£100,000 - £110,000 + EQUITY** **LONDON - HYBRID** This role is with a rapidly growing FinTech Scale up who are looking to build out their data engineering team. They are looking to hire a Lead Data Engineer to work closely with the Engineering Manager and Architects to take their data platform & capabilityto the next...


  • London, United Kingdom Harnham - Data & Analytics Recruitment Full time

    **MID-LEVEL DATA ENGINEER** **£60,000 - £70,000 + BENEFITS** **LONDON (FLEXI-WORKING)** Are you a Data Engineer looking for an exciting challenge? Want to become a pivotal part of a globally recognised insurance company? If becoming a Principal Data Engineer is the role you have been looking for, read on to discover more. **THE COMPANY**: Harnham is...

  • Lead Data Engineer

    4 weeks ago


    London, United Kingdom Harnham - Data & Analytics Recruitment Full time

    **LEAD DATA ENGINEER £70,000 - £75,000 + BENEFITS HYBRID - LONDON OR MAIDENHEAD** Are you a Senior Data Engineer looking for an exciting new role working within a team of likeminded individuals? Do you want to become an integral part of a team working within a betting company? If this is the role you have been searching for read on! **THE...

  • Senior Data Engineer

    4 weeks ago


    London, United Kingdom Harnham - Data & Analytics Recruitment Full time

    **SENIOR DATA ENGINEER** **LONDON BASED** **£90,000-100,000 PER ANNUM** This UK-based Fintech are searching for a new Senior Data Engineer to help architect and build a new data platform in AWS. You will also be responsible for liaising with stakeholders and influencing future business decisions. **THE COMPANY** This growing UK-based Fintech company...

  • Data Developer

    4 weeks ago


    London, United Kingdom Data Intellect Full time

    Welcome to Data Intellect . Providing technical leadership on client projects Leading technical teams to deliver projects on time and on budget Developing, testing, supporting and maintaining all code for client applications Analysing data and creating reports for clients, incorporating all findings Mentorship, training and potentially management...

  • Data Engineer

    4 weeks ago


    London, United Kingdom Harnham - Data & Analytics Recruitment Full time

    **DATA ENGINEER - WEB SCRAPER** **SALARY: TBC** **LONDON** Are you looking to work within the Virtual Art space? This company is looking for a skilled Data Engineer with Web Scraping capabilities, to gather vital data to ensure the business can scale up and reach new heights. **THE COMPANY** This company is extremely data-driven focusing on providing...

  • Senior Consultant

    2 weeks ago


    London, United Kingdom Data Idols Full time

    Senior Analytics Consultant Brighton - Hybrid/Remote Data Idols are working with a Google Cloud Consultancy that helps organisations transform their data analytics functions using modern technologies. With clients across the globe, they are looking for a Senior Analytics Consultant to join their expanding team. As a Senior Analytics Consultant, you...

  • Senior Consultant

    3 weeks ago


    London, United Kingdom Data Idols Full time

    Senior Analytics Consultant Brighton - Hybrid/Remote Data Idols are working with a Google Cloud Consultancy that helps organisations transform their data analytics functions using modern technologies. With clients across the globe, they are looking for a Senior Analytics Consultant to join their expanding team. As a Senior Analytics Consultant, you...


  • London, United Kingdom ON Data Staffing Full time

    We are seeking a dynamic and experienced Databricks professional to join our client, a growing startup consultancy, as a Lead Data Engineer. As Lead Data Engineer, you will play a pivotal role in driving the success of our client's Databricks practice through your expertise in presales, strong technical acumen, consulting skills, and effective team...

  • Lead Data Engineer

    4 weeks ago


    London, United Kingdom ON Data Staffing Full time

    We are seeking a dynamic and experienced Databricks professional to join our client, a growing startup consultancy, as a Lead Data Engineer.As Lead Data Engineer, you will play a pivotal role in driving the success of our client's Databricks practice through your expertise in presales, strong technical acumen, consulting skills, and effective team...