Data Engineer

3 weeks ago


City of London, United Kingdom N P Associates Full time

**Data Engineer**

This is an exciting opportunity for a Data Engineer to join a team of other highly skilled data professionals responsible for managing and enhancing the data estate.

**Purpose of Role**:
You will be part of a team responsible for data as a whole within the organisation, including data pipelines moving data from various on premise and cloud based source systems into the Azure based data platform for BI and analytics.

The Azure data platform consists of a number of Azure services including Data Factory, Data Lake Storage, Synapse Analytics, and Power BI.

This role also offers the opportunity to work with other database/data platforms like MongoDB and Splunk.

**Responsibilities**:

- Designing and implementing data pipelines using Azure Data Factory, Spark, SQL feeding data from source systems through to the data lake, data warehouse and Power BI
- Monitoring and supporting the existing data pipelines
- Design and evolution of the Data Lake
- Data Warehouse Schema modelling (Kimball/Star)
- ELT Data transformation into the Data Warehouse Schema
- Proficient in database performance analysis and tuning, particularly with Synapse SQL Pools.
- Developing 'best practice' standards within the data team.
- Continual learning about latest technologies in the data space.
- Maintaining a consistently high level of service delivery for incidents, requests, and changes.
- Maintain security standards across the data platform.

**Skills & Experienced Required**:

- Good communication skills with technology staff and end users.
- Strong technical knowledge of the Azure Data Platform including Data Factory, Data Lake, Synapse Analytics
- Strong knowledge of Azure Synapse Analytics SQL Pools and how it differs to SQL Server for performance reasons
- Strong SQL skills (SQL, T-SQL, PL/SQL)
- Scripting skills (Python, PowerShell)
- Data Lake Design - optimal layout for performance, security etc
- Experience in managing and migrating large volumes of data. Compression, partitioning and ETL
- Experience of other data cloud technologies - AWS, Azure, Oracle Cloud.

**Desirable**:

- Spark (Databricks/Synapse Spark Pools)
- MongoDB
- Splunk
- Experience of Oracle Cloud (OCI)
- Delta Lake
- Experience of working in an Agile/SCRUM environment
- A desire to learn and understand more about the Business environment in which you work.
- Technical knowledge of SQL Server to 2019 and Oracle to 19c.


  • Data Engineer

    1 month ago


    City of London, United Kingdom Data Idols Full time

    **Senior Data Engineer** Data Idols are proud to be working with an iconic UK brand to assist them in hiring a Senior Data Engineer as they undertake a digital and data-driven journey that will transform the business. Data is at the heart of the organisation and forms a key partof their future plans. **The Opportunity** As a Senior Data Engineer you will be...

  • Lead Data Engineer

    1 month ago


    City of London, United Kingdom Data Idols Full time

    **Lead Data Engineer** Data Idols are proud to be working with an iconic UK brand to assist them in hiring a Lead Data Engineer as they undertake a digital and data-driven journey that will transform the business. Data is at the heart of the organisation and forms a key part oftheir future plans. **Skills and Experience**: - Awareness and understanding of...

  • Data Architect

    4 weeks ago


    City of London, United Kingdom Data Idols Full time

    **Data Architect** Data Idols are working with one of the UK's biggest e-commerce organisations on a newly created data architect position that will drive their digital transformation. Home to some of the best-known fashion and homewares brands they have seen strong growthover the past 5 years and continue to learn more about their customers through...


  • City of London, United Kingdom Data Idols Full time

    **Principal Product Data Scientist** We're looking to recruit a Principal Data Scientist to lead experimentation on a greenfield project for one of the worlds best known online companies. If you have a very strong background in statistics and A/B testing we are keen to talk with you. Offeringa very competitive salary with flexible working across Europe this...

  • DevOps Engineer

    1 month ago


    City of London, United Kingdom Data Idols Full time

    **DevOps Engineer** Data Idols are working with a technology business helping their customers improve their financial health. They are an established and trusted provider of financial solutions and their platform is used by thousands of people everyday. **The Opportunity** As a DevOps Engineer you will have ownership of the platform and will lead the...


  • London, United Kingdom Data Idols Full time

    Senior Data Engineer Location: London (Hybrid) Data Idols are working with one of the fast-growing companies in the logistics sector that are making the world a greener place. They are looking for a Senior Data Engineer to join their team within the Data Space. As a Senior Data Engineer, you will work alongside different data teams to design,...


  • London, United Kingdom Data Idols Full time

    Senior Data Engineer Location: London (Hybrid) Data Idols are working with one of the fast-growing companies in the logistics sector that are making the world a greener place. They are looking for a Senior Data Engineer to join their team within the Data Space. As a Senior Data Engineer, you will work alongside different data teams to design,...


  • London, United Kingdom Data Idols Full time

    Job Description Senior Data Engineer Location: London (Hybrid) Data Idols are working with one of the fast-growing companies in the logistics sector that are making the world a greener place. They are looking for a Senior Data Engineer to join their team within the Data Space. As a Senior Data Engineer, you will work alongside different data teams to...

  • Gcp Data Engineer

    1 month ago


    London, United Kingdom Data Idols Full time

    **GCP Data Engineer** Would you like to build enterprise scale data solutions for a global organisation that reaches 400 million users monthly? As a GCP Data Engineer, you will be involved in delivering solutions from initial concept to production. **The Opportunity** Working for a well known brand you will be responsible for all aspects of the data...


  • London, United Kingdom Abs Data Full time

    Lead Data Engineer Do you enjoy creating valuable insights with data? Do you want to play a key role in transforming our firm into an agile organization? Being agile will make us more responsive, more adaptable, and ultimately more innovative. We’re looking for a Data Engineer to: ~ transform data into valuable insights that inform business...


  • London, United Kingdom Abs Data Full time

    Regulatory Reporting, Big Data Software Engineer We’re looking for software engineers to: provide technology solutions that will solve business problems and strengthen our position as digital leaders in financial services work with our Regulatory Reporting team to translate business and external regulatory needs using Behavior Driven Development...

  • Data Engineer

    3 days ago


    City of London, United Kingdom Harrington Starr Full time

    Data EngineerOn-Premise / Excel / SQLWe have partnered with a leading client based in Central London who are looking to increase their data engineering team.Role:You will be working on a large volume of various financial reporting data sources. The client here is using on-premise databases including Excel services & SQL.You will be working to gather data...

  • Data Engineer

    4 weeks ago


    London, United Kingdom Harnham - Data & Analytics Recruitment Full time

    **DATA ENGINEERING £70,000 - £80,000 + BONUS FLEXIBLE - LONDON** Are you a Data Engineer with a strong background in Data Engineering supporting the Data Analytics and Data Science team? Want to become part of a growing insurance company going through some exciting new changes? Read on! **THE COMPANY**: Harnham is partnered with an exciting insurance...

  • Data Engineer

    2 weeks ago


    City Of London, United Kingdom i3 Full time

    Data Engineer -SQL, Power BI, Analytics - PermanentData EngineerSQL, Power BI, Data AnalyticsInsuranceCity, London/ Hybrid (3 in the office)PermanentCompetitive SalaryA fantastic opportunity has arisen for an experienced Data Engineer ideally with previous Insurance background, 2-5 years of work experience with Analytical users and a good exposure to Cloud...

  • Data Engineer

    2 weeks ago


    City Of London, United Kingdom i3 Full time

    Data Engineer -SQL, Power BI, Analytics - PermanentData EngineerSQL, Power BI, Data AnalyticsInsuranceCity, London/ Hybrid (3 in the office)PermanentCompetitive SalaryA fantastic opportunity has arisen for an experienced Data Engineer ideally with previous Insurance background, 2-5 years of work experience with Analytical users and a good exposure to Cloud...

  • Data Engineer

    2 weeks ago


    City Of London, United Kingdom i3 Full time

    Data Engineer -SQL, Power BI, Analytics - Permanent Data Engineer SQL, Power BI, Data Analytics Insurance City, London/ Hybrid (3 in the office) Permanent Competitive Salary A fantastic opportunity has arisen for an experienced Data Engineer ideally with previous Insurance background, 2-5 years of work experience with Analytical users and a good exposure to...

  • Data Engineer

    5 days ago


    City of London, United Kingdom I3 Full time

    Data Engineer -SQL, Power BI, Analytics - PermanentThe following information provides an overview of the skills, qualities, and qualifications needed for this role.Data EngineerSQL, Power BI, Data AnalyticsInsuranceCity, London/ Hybrid (3 in the office)PermanentCompetitive SalaryA fantastic opportunity has arisen for an experienced Data Engineer ideally with...

  • Data Engineer

    4 weeks ago


    London, United Kingdom Harnham - Data & Analytics Recruitment Full time

    DATA ENGINEER **Salary: £50,000 - £70,000** **LONDON** This is an extremely technically driven company offering many benefits. They are data-driven and are situated globally, streaming, and using data to best help their customer. **THE COMPANY** This global data company is heavily driven by Data and Analytics and is looking for a Data Engineer to...

  • Data Engineer

    2 weeks ago


    City of London, United Kingdom i3 Full time

    Job DescriptionData Engineer -SQL, Power BI, Analytics - PermanentData EngineerSQL, Power BI, Data AnalyticsInsuranceCity, London/ Hybrid (3 in the office)PermanentCompetitive SalaryA fantastic opportunity has arisen for an experienced Data Engineer ideally with previous Insurance background, 2-5 years of work experience with Analytical users and a good...


  • London, United Kingdom ON Data Staffing Full time

    We are seeking a dynamic and experienced Databricks professional to join our client, a growing startup consultancy, as a Lead Data Engineer. As Lead Data Engineer, you will play a pivotal role in driving the success of our client's Databricks practice through your expertise in presales, strong technical acumen, consulting skills, and effective team...