Lead Engineer/Data Engineer
1 month ago
Location: London, Potters Bar, Bristol or Isle of Man (hybrid working options available)
The Lead Data Engineer is a hands-on technical role responsible for designing, developing, and maintaining data pipelines within the IT department. The pipelines will be realised in a modern lake environment and the engineer will collaborate in cross-functional teams to gather requirements and develop the conceptual data models. This role plays a crucial part in driving data-driven decision-making across the organisation, ensuring data availability, quality, and accessibility for various business needs.
This is not a line management role, but you will play a key part in guiding and upskilling more junior data engineers and setting data standards and guidelines.
Data Pipeline Development
Design, model, develop and maintain data pipelines to ingest, store, process, and present data.
Ensure data quality, accuracy, and consistency.
Collaborate with data architects to ensure data pipelines align with the overall data architecture strategy.
Data Transformation and Integration
Perform data transformation tasks, including data cleansing, enrichment, and aggregation, to prepare data for analytics and reporting.
Integrate data from structured and unstructured sources, ensuring compatibility and alignment with data models and business requirements.
Automate data transformation processes to improve efficiency.
Data Quality Assurance
Implement and maintain data quality checks and validation processes to identify and resolve data anomalies and errors.
Monitor data pipelines for data quality issues and implement data quality improvements.
Collaborate with business stakeholders to define data quality requirements.
Data Modelling and Schema Design
Collaborate with data architects and data scientists to design and implement data models, schemas, and structures.
Ensure that data models support business reporting and analytics needs while optimizing query performance.
Maintain data dictionaries and metadata to document data structures and relationships.
Performance Optimization
Optimize data storage, retrieval, and query performance by implementing indexing, partitioning, and caching strategies.
Monitor data processing performance and address bottlenecks as they arise.
Stay updated with best practices in data processing performance tuning.
Create and maintain documentation for data pipelines, data transformation processes, and data integration procedures.
Foster a culture of knowledge sharing within the data engineering team and across the organization.
Collaborate effectively with cross-functional teams, data stakeholders, and business units to understand data requirements and deliver data solutions that meet business needs.
Communicate technical concepts and data solutions to non-technical stakeholders in a clear and understandable manner.
Extensive experience in data engineering, including designing and developing data pipelines for retrieval / ingestion / presentation / semantics in an Azure environment.
Strong DataBricks, SQL, Python, Power BI
Data acquisition from various data sources including Salesforce, API, XML, json, parquet, flat file systems and relational data.
Effective communication and collaboration skills to work with cross-functional teams and gather data requirements.
Skills in data modelling (both structured and unstructured data) working directly with the business & data scientists.
Ability to optimise data solutions for performance, scalability, and efficiency.
Experience in a financial corporation
Experience with Spark clusters, both elastic permanent and transitory clusters
Familiarity with data governance, data security, and compliance requirements.
-
Electrical engineer engineer
1 month ago
Bromley, Greater London, United Kingdom The Engineer Full timeJob Title: Electrical Engineer Job Type: Permanent, Full time 00 pm, Monday to Friday Augustus Martin has been closely aligned with many of the UK's leading retailers, brands and out-of-home businesses for over 50 years. With a deep understanding of our clients' businesses and an unrivalled capability to create, manufacture and deliver all forms of...
-
Heating Engineer
1 month ago
London, United Kingdom The Engineer Full timeJob Title: Heating Engineer Location: London Salary: £320 per day Job type: Perm / Part time 3 days a week With over 30 years' experience, Everflame Heating Ltd provide both commercial and domestic utility services, covering all heating requirements. We are looking for an experienced Breakdown/Gas Safety Engineer to join our fantastic team 3...
-
Electrical Facilities Engineer
1 month ago
London, United Kingdom The Engineer Full timeJob Title: Electrical Facilities Engineer Location: London, E3 3PB Salary: £37,000 per annum. Job type: Full Time, Permanent About Us: Augustus Martin has been closely aligned with many of the UK's leading retailers, brands and out-of-home businesses for over 50 years. With a deep understanding of our clients' businesses and an unrivalled...
-
HIU Service Engineer
4 days ago
London, United Kingdom The Engineer Full timeHIU Service Engineer / Plumbing and Heating Engineer As a HIU Service Engineer, you will be required to service and repair various types of HIUs. This includes the commissioning of heating circuits, setting up Domestic Hot Water for resident use, application of efficiency settings and the ability to carry out first time repairs. At Elevate, our core values...
-
Heating Engineer
1 month ago
London, United Kingdom The Engineer Full timePerm / Part time 3 days a week With over 30 years' experience, Everflame Heating Ltd provide both commercial and domestic utility services, covering all heating requirements. We are looking for an experienced Breakdown/Gas Safety Engineer to join our fantastic team 3 days week. All work pre booked appointments in residential properties for one of...
-
Service / Breakdown Engineer
1 week ago
London, United Kingdom The Engineer Full timeWith over 30 years' experience, Everflame Heating Ltd provide both commercial and domestic utility services, covering all heating requirements. We are looking for an experienced Service / Breakdown Engineer to join our fantastic team. The Role: Working in and around London area. (All north of the river) All work pre booked appointments in residential...
-
Junior Electronics Development Engineer
1 week ago
London, United Kingdom The Engineer Full timeJunior Electronics Development Engineer The starting salary is £43,227 rising to £48,456, which includes allowances totalling £2,841. The salary is broken down as £40,386 basic salary, which will increase annually until you reach the top of the scale £45,615. Plus, a location allowance of £1,841 and a non-pensionable allowance of £1,000. ...
-
Field Service Engineer
1 week ago
London, United Kingdom The Engineer Full timeWith over 30 years' experience, Everflame Heating Ltd provide both commercial and domestic utility services, covering all heating requirements. We are looking for an experienced Service / Breakdown Engineer to join our fantastic team. All work pre booked appointments in residential properties for one of London's largest private landlords. Gas safety...
-
Lead Engineer/Data Engineer
1 month ago
London, United Kingdom Abs Data Full timeLead Data Engineer Do you enjoy creating valuable insights with data? Do you want to play a key role in transforming our firm into an agile organization? Being agile will make us more responsive, more adaptable, and ultimately more innovative. We’re looking for a Data Engineer to: ~ transform data into valuable insights that inform business...
-
Lead Data Engineer
1 month ago
City of London, United Kingdom Data Idols Full time**Lead Data Engineer** Data Idols are proud to be working with an iconic UK brand to assist them in hiring a Lead Data Engineer as they undertake a digital and data-driven journey that will transform the business. Data is at the heart of the organisation and forms a key part oftheir future plans. **Skills and Experience**: - Awareness and understanding of...
-
Electrical Engineer
1 month ago
Bromley, Greater London, United Kingdom The Engineer Full timeJob Title: Electrical Engineer Location: Bromley-by-Bow, London Salary: £40,000 to £45,000 per annum DOE Job Type: Permanent, Full time Working Hours: 08:30 am to 17.00 pm, Monday to Friday About Us: Augustus Martin has been closely aligned with many of the UK's leading retailers, brands and out-of-home businesses for over 50 years. With a...
-
Sub-Contract Heating Installer
6 days ago
London, United Kingdom The Engineer Full timeSub-Contract Heating Installer / Gas Engineer Job Title: Sub-Contract Heating Installer Location: London Salary: Highly competitive daily rate Job Type: Part time, permanent With over 30 years' experience, Everflame Heating Ltd provide both commercial and domestic utility services, covering all heating requirements. We are looking for a sub-contract...
-
Fire & Security Engineer - Service
2 days ago
London, United Kingdom The Engineer Full timeJob Type: Full-Time, Permanent. They have been providing electrical and security services since 1972, over the years it has expanded to be an integrated facility solutions provider providing solutions in all areas of the corporate, domestic and elite sectors. The company provide solutions Nationally to & for: Road, Rail & Airport transportation hubs,...
-
Energy and Chemicals Risk Engineer
1 month ago
London, United Kingdom The Chemical Engineer Full timeInsurance Sector £70 to £115,000 PA + Quality Benefits Our client is a world leader in the energy, chemicals, and petroleum insurance sector, providing risk engineering services across the globe, for clients in all hazardous segments, from oil & gas exploration and production, through mid-stream, refining to downstream petrochemicals. The business is...
-
Fire and Security Engineer
2 days ago
London, United Kingdom The Engineer Full timeJob Type: Full-Time, Permanent. Location: London, ideally based no more than 1hr from W1J 6BS North, East, south or West. Salary: Negotiable dependent upon experience and ability. About The Company: They have been providing electrical and security services since 1972, over the years it has expanded to be an integrated facility solutions provider...
-
Data Engineering Lead Engineer
2 weeks ago
London, United Kingdom ON Data Staffing Full timeWe are seeking a dynamic and experienced Databricks professional to join our client, a growing startup consultancy, as a Lead Data Engineer. As Lead Data Engineer, you will play a pivotal role in driving the success of our client's Databricks practice through your expertise in presales, strong technical acumen, consulting skills, and effective team...
-
Lead Data Engineer
1 month ago
London, United Kingdom Harnham - Data & Analytics Recruitment Full time**LEAD DATA ENGINEER £70,000 - £75,000 + BENEFITS HYBRID - LONDON OR MAIDENHEAD** Are you a Senior Data Engineer looking for an exciting new role working within a team of likeminded individuals? Do you want to become an integral part of a team working within a betting company? If this is the role you have been searching for read on! **THE...
-
Lead Data Engineer
1 month ago
London, United Kingdom Harnham - Data & Analytics Recruitment Full time**LEAD DATA ENGINEER** **£100,000 - £110,000 + EQUITY** **LONDON - HYBRID** This role is with a rapidly growing FinTech Scale up who are looking to build out their data engineering team. They are looking to hire a Lead Data Engineer to work closely with the Engineering Manager and Architects to take their data platform & capabilityto the next...
-
Lead Risk Management Analyst
2 days ago
London, United Kingdom The Engineer Full timeLocation: Various - We offer a range of flexible working arrangements - please speak to your recruiter about the options for this role. Salary: £60,000+ Depending on experience Lead on developing the risk management data strategy; identifying potential data sources and approaches to connecting and exploiting the data to support risk analysis Conduct...
-
Lead Cyber Risk Analyst
2 days ago
London, United Kingdom The Engineer Full timeLocation: Various - We offer a range of flexible working arrangements - please speak to your recruiter about the options for this role. Salary: £60,000+ Depending on experience What you'll be doing: Lead on developing the risk management data strategy; identifying potential data sources and approaches to connecting and exploiting the data to...