Lead Data Engineer

3 months ago

Norwich, United Kingdom Baldwin Risk Partners Full time

Why MSI? We thrive on solving challenges. As a leading MGA, MSI combines deep underwriting expertise with insurer risk capacity to create specialized insurance solutions that empower distribution partners to meet customers’ unique needs. We have a passion for crafting solutions for the important risks facing individuals and businesses. We offer an expanding suite of products – from fully-digital embedded renters coverage to high-value homeowners insurance to sophisticated commercial coverages, such as cyber liability and habitational property – delivered through agents, brokers, wholesalers and other brand partners.
 
Our partners and customers count on us to deliver exceptional service through a dedicated team that makes rapid resolutions a priority. We simplify the insurance experience through our advanced technology platform that supports every phase of the policy lifecycle. Bring on your challenges and let us show you how we build insurance better.

The Lead Data Engineer will be responsible for, overseeing the design, building and optimization of

our data orchestration and data pipeline architecture, as well as optimizing data collection and flow for cross functional teams.

The ideal candidate is an experienced technical oriented Lead Data Engineer who enjoys data orchestration and optimizing data systems and building them from the ground up. This role requires a deep understanding of data management / data engineering principles as they apply to scalability, and design of dynamic flexible data orchestration patterns. Additionally, the Lead Data Engineer is responsible for mentoring junior engineers and providing technical guidance and a ‘design thinking’ perspective to the team members.

They must be self-directed and comfortable supporting the data needs of multiple teams, systems and products. The right candidate will be excited by the prospect of contributing to the design of our company’s data architecture to support our next generation of products and data initiatives. You will be equally comfortable hands on building data systems as you are mentoring more junior team members.

Principal Responsibilities:

Maintaining and updating our Data Engineering Architecture.Designing and implementing ELT and ETL processes.Create and maintain optimal data orchestration architecture to support data initiatives.Provide thought leadership that helps the team improve implementation approaches that drive increased velocity of deliveryAssemble large, complex data sets that meet functional / non-functional business requirements.Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.Build the infrastructure required for optimal extraction, transformation, and ingestion of data from a wide variety of data sources using SQL, Azure Data Factory, Spark / Synapse technologies.Support analytics tools that utilize the data pipeline to provide actionable insights into key business performance metrics.Work with stakeholders including the Executive, Product, Data and Technology teams to assist with data-related technical issues and support their data infrastructure needs.Create data tools for analytics and data scientist team members that assist them in executing and optimizing data projects.Work with data and analytics experts to strive for greater functionality in our data systems.Collaborating with colleagues for the purpose of collecting and structuring data.

Collect, audit, compile, and validate data from multiple sources.

Communicate internally and with clients externally to collect and validate data as well as answer questions regarding data.Apply advanced knowledge and understanding of concepts, principals, and technical capabilities to manage a wide variety of projects.Recommends new practices, processes and procedures.Provides solutions that may set precedent or have significant impact.Build automation and additional efficiencies into manual efforts.

Education, Experience, Skills and Abilities Requirements:

Bachelor’s degree in related field preferred, equivalent years’ experience considered.At least seven to ten years of data related or analytical work experience in a Data Engineer role, preferably three of those within the Azure ecosystem.Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of data platforms.Advanced understanding of and experience implementing data lakes, lake houses.Advanced understanding and experience with file storage layer management in data lake environment, including parquet and delta file formats.Solid experience with SPARK (PySpark) language, and data processing techniques.Solid Understanding of and experience with AZURE SYNAPSE tools and services.Some knowledge of Python preferred.Strong analytic skills related to working with structured, semi-structured, and unstructured datasets and blob storage.Build processes supporting data transformation, data structures, metadata, dependency and workload management.A successful history of manipulating, processing and extracting value from large disconnected datasets.Strong project management and organizational skills.Experience supporting and working with cross-functional teams in a dynamic environment.Insurance industry experience preferred.

Special Working Conditions:

Fast paced, multi-tasking environment.

Important Notice:

This position description is intended to describe the level of work required of the person performing in the role and is not a contract. The essential responsibilities are outlined; other duties may be assigned as needs arise or as required to support the organization. All requirements may be modified to reasonably accommodate physically or mentally challenged colleagues.

#LI-BM1

#LI-REMOTE

Click for some insight into our culture

  • Senior Data Engineering Lead

    3 weeks ago

    Norwich, Norfolk, United Kingdom Aviva Full time

    Data Engineering Technical LeadAbout the roleAviva is seeking a skilled Data Engineering Technical Lead to drive the development of data engineering solutions. The ideal candidate will have expertise in designing scalable data pipelines, data warehousing, and big data technologies.Key responsibilitiesLead the development of data engineering solutions,...

  • Data Engineering Lead

    3 weeks ago

    Norwich, Norfolk, United Kingdom Aviva Full time

    As a Principal Data Engineer at Aviva, you will be an integral part of our Data Engineering team, responsible for designing and implementing end-to-end data engineering pipelines. Your expertise in cloud platforms, data ingestion, and integration will be crucial in enhancing our data operations.We are seeking a skilled professional with a strong background...

  • Data Engineering Lead

    7 days ago

    Norwich, Norfolk, United Kingdom Aviva Full time

    About the JobAs a seasoned technical expert, we are seeking a highly skilled Data Engineering Technical Lead to join our team. This role offers a unique opportunity to drive success in data lake/lake house architecture and develop data pipelines using pyspark and SQL.Key ResponsibilitiesProvide technical expertise and delivery capability across development &...

  • Data Engineer

    3 weeks ago

    Norwich, Norfolk, United Kingdom Boston Hale Full time

    Data Engineer Role at Boston HaleWe are seeking a highly skilled Data Engineer to join our team at Boston Hale. As a Data Engineer, you will be responsible for designing, building, and maintaining large-scale data processing systems, including data pipelines, ETL processes, and data warehouses.In this role, you will:Design and Implement Data Pipelines:...

  • Strategic Data Engineering Director

    7 days ago

    Norwich, Norfolk, United Kingdom Aviva Full time

    Job SummaryThis role involves leading data engineering solutions within the Platform team to deliver a cost-effective and leading data engineering service.The successful candidate will support strategic programmes, provide guidance on delivery of data engineering,and collaborate with areas like Group Architecture and key business stakeholders.

  • ICT Data Engineer

    2 months ago

    Norwich, United Kingdom CV-Library Full time

    Norfolk Wildlife Trust are seeking an ICT Data Engineer to work as a key member of our ICT Team to provide the management, co-ordination and development of the Trust's data and systems. ICT Data Engineer Salary: £25,553 - £28,146 per annum Contract type: 1 year fixed term Working hours: Full time Location: Norfolk Wildlife Trust, Bewick...

  • ICT Data Engineer

    4 days ago

    Norwich, United Kingdom CV-Library Full time

    Norfolk Wildlife Trust are seeking an ICT Data Engineer to work as a key member of our ICT Team to provide the management, co-ordination and development of the Trust's data and systems. ICT Data Engineer Salary: £25,553 - £28,146 per annum Contract type: 1 year fixed term Working hours: Full time Location: Norfolk Wildlife Trust, Bewick...

  • Data Engineer

    1 month ago

    Norwich, Norfolk, United Kingdom Boston Hale Full time

    We are seeking a highly skilled Data Engineer to join our team at Boston Hale. As a Data Engineer, you will be responsible for designing, building, and maintaining large-scale data systems to support our business operations. You will work closely with cross-functional teams to develop and implement data-driven solutions that drive business growth and...

  • Senior Designer of Data Engineering Services

    7 days ago

    Norwich, Norfolk, United Kingdom Aviva Full time

    Data Engineering LeadThis is an exciting opportunity to take your data engineering skills to the next level and lead the design of data engineering solutions within the Platform team. You will work closely with cross-functional teams to deliver innovative data engineering solutions that meet the evolving needs of our customers.You will also provide thought...

  • Lead Data Solutions Architect

    1 week ago

    Norwich, Norfolk, United Kingdom Aviva Full time

    About the RoleWe are seeking an experienced Senior Designer of Data Engineering Services to join our team. As a member of the Platform team, you will be responsible for designing and delivering data engineering solutions that meet the needs of our customers.The ideal candidate will have strong Data Analysis knowledge and experience, as well as expertise in...

  • Data Engineering Specialist

    7 days ago

    Norwich, Norfolk, United Kingdom Boston Hale Full time

    Job Description:We are seeking a highly skilled Data Engineer to join our team at Boston Hale. As a key member of our analytics department, you will be responsible for managing the end-to-end analytics process, from data cleaning to delivering insights.Your primary focus will be on implementing end-to-end ETL processes to clean and implement data from...

  • Chief Data Analytics Lead

    2 weeks ago

    Norwich, Norfolk, United Kingdom Aviva Full time

    Aviva's Senior Data Science PositionAt Aviva, we're investing in cutting-edge data science to shape the future of insurance. We're seeking an experienced Lead Data Scientist to join our high-profile team and lead the way in data-driven decision making.This exciting opportunity offers a chance to work with one of the richest insurance datasets in the world,...

  • Data Engineering Leader for Cloud and Data Innovation

    2 weeks ago

    Norwich, Norfolk, United Kingdom Aviva Full time

    At Aviva, data serves as the lifeblood that propels our organizations forward. As a Principal Data Engineer within our esteemed Data Engineering team, you will be an indispensable force, seamlessly integrating into Aviva Quantum, our global hub for Data Science Excellence.Job OverviewWe are seeking a highly experienced Principal Data Engineer to join our...

  • Software Engineering Lead

    2 weeks ago

    Norwich, Norfolk, United Kingdom Aviva Full time

    Transform Customer Data with UsWe are seeking a talented Software Engineering Lead to join our dynamic team at Aviva, based in Norwich. As a key member of our Customer Data Platforms team, you will play a vital role in shaping the future of our data estate and driving innovation in customer interactions.About the Role:This is an exciting opportunity for a...

  • Senior Data Engineer Position

    7 days ago

    Norwich, Norfolk, United Kingdom Aviva Full time

    About the RoleWe're seeking a talented Data Engineering Technical Lead to join our growing team. If you have experience working as a senior data engineer in data lake/lake house architecture and hands-on experience of developing data pipelines using pyspark and SQL, we'd love to hear from you.ResponsibilitiesProvide technical input into solution designs and...

  • Software Engineering Lead

    2 weeks ago

    Norwich, Norfolk, United Kingdom Aviva Full time

    Senior Software EngineerWe are seeking an experienced Senior Software Engineer to join our dynamic team at Aviva. As a key member of our Customer Data Platform, you will be responsible for leading the development of innovative software solutions and guiding the engineering principles of your team.This role is based in Norwich, where you will have the...

  • Principal Engineer

    7 days ago

    Norwich, Norfolk, United Kingdom oilandgasjobsearch Full time

    About the RoleWe are seeking an experienced Principal Engineer - Data to join our team at oilandgasjobsearch. As a key member of the software development team, you will be responsible for leading design discussions, driving team efficiency through Agile and DevOps methodologies, and extending the IBM ELM software suite using product APIs.Key...

  • Software Engineering Team Lead

    3 days ago

    Norwich, Norfolk, United Kingdom Aviva Full time

    Our Aviva team is seeking a highly skilled Software Engineering Team Lead to join our Customer Data Platforms division.This role resides in the Customer Data team, which is at the core of Aviva's digital strategy and provides a single source of customer data across our Customer and Marketing, GI and IWR propositions.We're constantly striving to explore...

  • Data Governance Strategic Lead

    7 days ago

    Norwich, United Kingdom Cooper Lomaz Recruitment Ltd Full time

    Job Title: Data Governance Strategic LeadAbout Us:Cooper Lomaz Recruitment Ltd is partnering with an organisation at the forefront of leveraging data to drive business growth. We are seeking a seasoned Data Governance Strategic Lead with a strong background in data management and governance to contribute to shaping their data strategy.The Role:This pivotal...

  • Technical Data Engineering Role

    7 days ago

    Norwich, Norfolk, United Kingdom Aviva Full time

    About Our TeamWe're a dynamic team seeking a talented Data Engineering Technical Lead to drive success in data lake/lake house architecture and develop data pipelines using pyspark and SQL. If you're an experienced technical expert with a passion for driving success, we want to hear from you.Key ResponsibilitiesProvide technical expertise and delivery...