Lead Engineer/Data Engineer

1 month ago


South East London, United Kingdom Canada Life Full time

Location: London, Potters Bar, Bristol or Isle of Man (hybrid working options available)
The Lead Data Engineer is a hands-on technical role responsible for designing, developing, and maintaining data pipelines within the IT department. The pipelines will be realised in a modern lake environment and the engineer will collaborate in cross-functional teams to gather requirements and develop the conceptual data models. This role plays a crucial part in driving data-driven decision-making across the organisation, ensuring data availability, quality, and accessibility for various business needs.
This is not a line management role, but you will play a key part in guiding and upskilling more junior data engineers and setting data standards and guidelines.
Data Pipeline Development

Design, model, develop and maintain data pipelines to ingest, store, process, and present data.
Ensure data quality, accuracy, and consistency.
Collaborate with data architects to ensure data pipelines align with the overall data architecture strategy.
Data Transformation and Integration

Perform data transformation tasks, including data cleansing, enrichment, and aggregation, to prepare data for analytics and reporting.
Integrate data from structured and unstructured sources, ensuring compatibility and alignment with data models and business requirements.
Automate data transformation processes to improve efficiency.
Data Quality Assurance

Implement and maintain data quality checks and validation processes to identify and resolve data anomalies and errors.
Monitor data pipelines for data quality issues and implement data quality improvements.
Collaborate with business stakeholders to define data quality requirements.
Data Modelling and Schema Design

Collaborate with data architects and data scientists to design and implement data models, schemas, and structures.
Ensure that data models support business reporting and analytics needs while optimizing query performance.
Maintain data dictionaries and metadata to document data structures and relationships.
Performance Optimization

Optimize data storage, retrieval, and query performance by implementing indexing, partitioning, and caching strategies.
Monitor data processing performance and address bottlenecks as they arise.
Stay updated with best practices in data processing performance tuning.
Create and maintain documentation for data pipelines, data transformation processes, and data integration procedures.
Foster a culture of knowledge sharing within the data engineering team and across the organization.
Collaborate effectively with cross-functional teams, data stakeholders, and business units to understand data requirements and deliver data solutions that meet business needs.
Communicate technical concepts and data solutions to non-technical stakeholders in a clear and understandable manner.
Extensive experience in data engineering, including designing and developing data pipelines for retrieval / ingestion / presentation / semantics in an Azure environment.
Strong DataBricks, SQL, Python, Power BI
Data acquisition from various data sources including Salesforce, API, XML, json, parquet, flat file systems and relational data.
Effective communication and collaboration skills to work with cross-functional teams and gather data requirements.
Skills in data modelling (both structured and unstructured data) working directly with the business & data scientists.
Ability to optimise data solutions for performance, scalability, and efficiency.
Experience in a financial corporation
Experience with Spark clusters, both elastic permanent and transitory clusters
Familiarity with data governance, data security, and compliance requirements.



  • Bromley, Greater London, United Kingdom The Engineer Full time

    Job Title: Electrical Engineer Job Type: Permanent, Full time 00 pm, Monday to Friday Augustus Martin has been closely aligned with many of the UK's leading retailers, brands and out-of-home businesses for over 50 years. With a deep understanding of our clients' businesses and an unrivalled capability to create, manufacture and deliver all forms of...

  • Heating Engineer

    1 month ago


    London, United Kingdom The Engineer Full time

    Job Title:  Heating Engineer Location:  London Salary: £320 per day Job type: Perm / Part time 3 days a week With over 30 years' experience, Everflame Heating Ltd provide both commercial and domestic utility services, covering all heating requirements. We are looking for an experienced Breakdown/Gas Safety Engineer to join our fantastic team 3...


  • London, United Kingdom The Engineer Full time

    Job Title: Electrical Facilities Engineer Location: London, E3 3PB Salary: £37,000 per annum. Job type: Full Time, Permanent About Us: Augustus Martin has been closely aligned with many of the UK's leading retailers, brands and out-of-home businesses for over 50 years. With a deep understanding of our clients' businesses and an unrivalled...


  • London, United Kingdom The Engineer Full time

    HIU Service Engineer / Plumbing and Heating Engineer As a HIU Service Engineer, you will be required to service and repair various types of HIUs. This includes the commissioning of heating circuits, setting up Domestic Hot Water for resident use, application of efficiency settings and the ability to carry out first time repairs. At Elevate, our core values...

  • Heating Engineer

    1 month ago


    London, United Kingdom The Engineer Full time

    Perm / Part time 3 days a week With over 30 years' experience, Everflame Heating Ltd provide both commercial and domestic utility services, covering all heating requirements. We are looking for an experienced Breakdown/Gas Safety Engineer to join our fantastic team 3 days week. All work pre booked appointments in residential properties for one of...


  • London, United Kingdom The Engineer Full time

    With over 30 years' experience, Everflame Heating Ltd provide both commercial and domestic utility services, covering all heating requirements. We are looking for an experienced Service / Breakdown Engineer to join our fantastic team. The Role: Working in and around London area. (All north of the river) All work pre booked appointments in residential...


  • London, United Kingdom The Engineer Full time

    Junior Electronics Development Engineer The starting salary is £43,227 rising to £48,456, which includes allowances totalling £2,841. The salary is broken down as £40,386 basic salary, which will increase annually until you reach the top of the scale £45,615. Plus, a location allowance of £1,841 and a non-pensionable allowance of £1,000. ...


  • London, United Kingdom The Engineer Full time

    With over 30 years' experience, Everflame Heating Ltd provide both commercial and domestic utility services, covering all heating requirements. We are looking for an experienced Service / Breakdown Engineer to join our fantastic team. All work pre booked appointments in residential properties for one of London's largest private landlords. Gas safety...


  • London, United Kingdom Abs Data Full time

    Lead Data Engineer Do you enjoy creating valuable insights with data? Do you want to play a key role in transforming our firm into an agile organization? Being agile will make us more responsive, more adaptable, and ultimately more innovative. We’re looking for a Data Engineer to: ~ transform data into valuable insights that inform business...

  • Lead Data Engineer

    1 month ago


    City of London, United Kingdom Data Idols Full time

    **Lead Data Engineer** Data Idols are proud to be working with an iconic UK brand to assist them in hiring a Lead Data Engineer as they undertake a digital and data-driven journey that will transform the business. Data is at the heart of the organisation and forms a key part oftheir future plans. **Skills and Experience**: - Awareness and understanding of...

  • Electrical Engineer

    1 month ago


    Bromley, Greater London, United Kingdom The Engineer Full time

    Job Title: Electrical Engineer Location: Bromley-by-Bow, London Salary: £40,000 to £45,000 per annum DOE Job Type: Permanent, Full time Working Hours: 08:30 am to 17.00 pm, Monday to Friday About Us: Augustus Martin has been closely aligned with many of the UK's leading retailers, brands and out-of-home businesses for over 50 years. With a...


  • London, United Kingdom The Engineer Full time

    Sub-Contract Heating Installer / Gas Engineer Job Title: Sub-Contract Heating Installer Location: London Salary: Highly competitive daily rate Job Type: Part time, permanent With over 30 years' experience, Everflame Heating Ltd provide both commercial and domestic utility services, covering all heating requirements. We are looking for a sub-contract...


  • London, United Kingdom The Engineer Full time

    Job Type: Full-Time, Permanent. They have been providing electrical and security services since 1972, over the years it has expanded to be an integrated facility solutions provider providing solutions in all areas of the corporate, domestic and elite sectors. The company provide solutions Nationally to & for: Road, Rail & Airport transportation hubs,...


  • London, United Kingdom The Chemical Engineer Full time

    Insurance Sector £70 to £115,000 PA + Quality Benefits Our client is a world leader in the energy, chemicals, and petroleum insurance sector, providing risk engineering services across the globe, for clients in all hazardous segments, from oil & gas exploration and production, through mid-stream, refining to downstream petrochemicals. The business is...


  • London, United Kingdom The Engineer Full time

    Job Type: Full-Time, Permanent. Location: London, ideally based no more than 1hr from W1J 6BS North, East, south or West. Salary: Negotiable dependent upon experience and ability. About The Company: They have been providing electrical and security services since 1972, over the years it has expanded to be an integrated facility solutions provider...


  • London, United Kingdom ON Data Staffing Full time

    We are seeking a dynamic and experienced Databricks professional to join our client, a growing startup consultancy, as a Lead Data Engineer. As Lead Data Engineer, you will play a pivotal role in driving the success of our client's Databricks practice through your expertise in presales, strong technical acumen, consulting skills, and effective team...

  • Lead Data Engineer

    1 month ago


    London, United Kingdom Harnham - Data & Analytics Recruitment Full time

    **LEAD DATA ENGINEER £70,000 - £75,000 + BENEFITS HYBRID - LONDON OR MAIDENHEAD** Are you a Senior Data Engineer looking for an exciting new role working within a team of likeminded individuals? Do you want to become an integral part of a team working within a betting company? If this is the role you have been searching for read on! **THE...

  • Lead Data Engineer

    1 month ago


    London, United Kingdom Harnham - Data & Analytics Recruitment Full time

    **LEAD DATA ENGINEER** **£100,000 - £110,000 + EQUITY** **LONDON - HYBRID** This role is with a rapidly growing FinTech Scale up who are looking to build out their data engineering team. They are looking to hire a Lead Data Engineer to work closely with the Engineering Manager and Architects to take their data platform & capabilityto the next...


  • London, United Kingdom The Engineer Full time

    Location: Various - We offer a range of flexible working arrangements - please speak to your recruiter about the options for this role. Salary: £60,000+ Depending on experience Lead on developing the risk management data strategy; identifying potential data sources and approaches to connecting and exploiting the data to support risk analysis Conduct...


  • London, United Kingdom The Engineer Full time

    Location: Various - We offer a range of flexible working arrangements - please speak to your recruiter about the options for this role. Salary: £60,000+ Depending on experience What you'll be doing: Lead on developing the risk management data strategy; identifying potential data sources and approaches to connecting and exploiting the data to...