Current jobs related to Senior Data Integration Specialist with AWS and PySpark - London, Greater London - N Consulting Ltd

  • Python Data Engineer with PySpark Experience

    2 weeks ago

    London, Greater London, United Kingdom Joseph Harry Full time

    Python Data Engineer RoleWe are seeking a highly skilled Python Data Engineer to join our team at Joseph Harry. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining large-scale data processing systems using Python and PySpark.Key ResponsibilitiesDesign and implement data pipelines using Python and...

  • Senior AWS Data Engineer

    3 weeks ago

    London, Greater London, United Kingdom Two Circles Full time

    About Two CirclesTwo Circles is a data-driven sports marketing agency that works with some of the world's biggest sports organisations. We're passionate about using data to drive innovation and deliver exceptional results for our clients.Job Title: Senior AWS Data EngineerWe're seeking a talented Senior AWS Data Engineer to join our growing team and help us...

  • Senior AWS Data Engineer

    3 weeks ago

    London, Greater London, United Kingdom Two Circles Full time

    About Two CirclesTwo Circles is a data-driven sports marketing agency that works with some of the world's biggest sports organisations. We're passionate about using data to drive innovation and deliver exceptional results for our clients.Job Title: Senior AWS Data EngineerWe're seeking a talented Senior AWS Data Engineer to join our growing team and help us...

  • Senior Data Engineer

    4 weeks ago

    London, Greater London, United Kingdom Damco Solutions Full time

    Job Title: Senior Engineer, Data EngineeringJob Summary:We are seeking a highly skilled Senior Engineer, Data Engineering to join our team at Damco Solutions. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining large-scale data processing systems on AWS.Key Responsibilities:Design and implement...

  • Senior Data Engineer

    4 weeks ago

    London, Greater London, United Kingdom Damco Solutions Full time

    Job Title: Senior Engineer, Data EngineeringJob Summary:We are seeking a highly skilled Senior Engineer, Data Engineering to join our team at Damco Solutions. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining large-scale data processing systems on AWS.Key Responsibilities:Design and implement...

  • AWS Cloud Engineer

    2 weeks ago

    London, Greater London, United Kingdom Kantar Group Limited Full time

    Job Title: Senior Software EngineerWe are seeking a highly skilled Senior Software Engineer to lead the design and development of high-quality software solutions using Java, Python, and PySpark. The ideal candidate will have extensive experience with Azure services and a strong focus on developing production-grade software.Key Responsibilities:Design and...

  • Data Services Developer

    3 weeks ago

    London, Greater London, United Kingdom Kite Consulting Group Full time

    Senior Data Engineer - Pyspark - Financial ServicesWe are seeking a highly skilled Senior Data Engineer with expertise in Pyspark to join our team at Kite Consulting Group. As a Senior Data Engineer, you will be responsible for designing and implementing data pipelines and architectures on the AWS platform.Key Responsibilities:Design and implement data...

  • Data Services Developer

    3 weeks ago

    London, Greater London, United Kingdom Kite Consulting Group Full time

    Senior Data Engineer - Pyspark - Financial ServicesWe are seeking a highly skilled Senior Data Engineer with expertise in Pyspark to join our team at Kite Consulting Group. As a Senior Data Engineer, you will be responsible for designing and implementing data pipelines and architectures on the AWS platform.Key Responsibilities:Design and implement data...

  • AWS Cloud Engineer

    1 month ago

    London, Greater London, United Kingdom Kantar Group Limited Full time

    Unlock Insights and Drive Business GrowthKantar Media is a leading provider of audience measurement, consumer targeting, and advertising intelligence services. As a Senior Software Engineer, you will play a critical role in developing production-grade software solutions that unlock insights and drive business growth for our clients.Key Responsibilities:Lead...

  • AWS Cloud Engineer

    1 month ago

    London, Greater London, United Kingdom Kantar Group Limited Full time

    Unlock Insights and Drive Business GrowthKantar Media is a leading provider of audience measurement, consumer targeting, and advertising intelligence services. As a Senior Software Engineer, you will play a critical role in developing production-grade software solutions that unlock insights and drive business growth for our clients.Key Responsibilities:Lead...

  • AWS Cloud Engineer

    2 weeks ago

    London, Greater London, United Kingdom Kantar Group Limited Full time

    Unlock Insights and Drive Business GrowthKantar Media is a leading provider of audience measurement, consumer targeting, and advertising intelligence services. As a Senior Software Engineer, you will play a critical role in developing production-grade software solutions that unlock insights and drive business growth for our clients.Key Responsibilities:Lead...

  • AWS Cloud Engineer

    2 weeks ago

    London, Greater London, United Kingdom Kantar Group Limited Full time

    Unlock Insights and Drive Business GrowthKantar Media is a leading provider of audience measurement, consumer targeting, and advertising intelligence services. As a Senior Software Engineer, you will play a critical role in developing production-grade software solutions that unlock insights and drive business growth for our clients.Key Responsibilities:Lead...

  • Azure Data Engineer

    2 weeks ago

    London, Greater London, United Kingdom Prism Digital Full time

    Senior Azure Data Engineer | PySpark (Python) & Synapse | Tech for Good | Fully RemoteAbout the Role:We are seeking a highly skilled Senior Azure Data Engineer to join our team at Prism Digital. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining large-scale data platforms on Azure.Key...

  • Senior Data Scientist

    4 weeks ago

    London, Greater London, United Kingdom Harnham - Data & Analytics Recruitment Full time

    Senior Data ScientistWe are seeking a highly skilled Senior Data Scientist to join our team at Harnham - Data & Analytics Recruitment. As a Senior Data Scientist, you will be working with leading retailers in the FMCG / retail sector to improve their knowledge and understanding of their customer base.Key Responsibilities:Develop and implement advanced...

  • Senior Data Scientist

    4 weeks ago

    London, Greater London, United Kingdom Harnham - Data & Analytics Recruitment Full time

    Senior Data ScientistWe are seeking a highly skilled Senior Data Scientist to join our team at Harnham - Data & Analytics Recruitment. As a Senior Data Scientist, you will be working with leading retailers in the FMCG / retail sector to improve their knowledge and understanding of their customer base.Key Responsibilities:Develop and implement advanced...

  • AWS Cloud Engineer

    2 weeks ago

    London, Greater London, United Kingdom Kantar Group Limited Full time

    Job Title: Senior Software EngineerWe are seeking a highly skilled Senior Software Engineer to lead the design and development of high-quality, production-grade software solutions using Java, Python, and PySpark. The ideal candidate will have extensive experience with Azure services, including Azure Storage, and a strong understanding of data security,...

  • AWS Cloud Engineer

    2 weeks ago

    London, Greater London, United Kingdom Kantar Group Limited Full time

    Job Title: Senior Software EngineerWe are seeking a highly skilled Senior Software Engineer to lead the design and development of high-quality, production-grade software solutions using Java, Python, and PySpark. The ideal candidate will have extensive experience with Azure services, including Azure Storage, and a strong understanding of data security,...

  • Senior Azure Data Engineer

    2 weeks ago

    London, Greater London, United Kingdom Prism Digital Full time

    Job SummaryWe are seeking a highly skilled Senior Azure Data Engineer to join our team at Prism Digital. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining large-scale data pipelines on Azure.Key ResponsibilitiesDesign and implement data architectures on Azure Synapse and PySparkDevelop and...

  • Senior Data Architect

    3 weeks ago

    London, Greater London, United Kingdom Kite Consulting Group Full time £650

    Senior Data Engineer – PySpark, Python – Contract (Inside IR35)Our client, a global financial services organisation, is seeking a highly skilled Senior Data Engineer to join their team in London. This is a PAYE contract role offering an umbrella rate of up to £650 per day.Key Responsibilities:Design and enhance data systems and workflows that drive key...

  • Senior Data Architect

    3 weeks ago

    London, Greater London, United Kingdom Kite Consulting Group Full time £650

    Senior Data Engineer – PySpark, Python – Contract (Inside IR35)Our client, a global financial services organisation, is seeking a highly skilled Senior Data Engineer to join their team in London. This is a PAYE contract role offering an umbrella rate of up to £650 per day.Key Responsibilities:Design and enhance data systems and workflows that drive key...

Senior Data Integration Specialist with AWS and PySpark

2 months ago

London, Greater London, United Kingdom N Consulting Ltd Full time

Job Title:

AWS Data Engineer with PySpark and Python Expertise

Location:

Remote

Contract:

6 months

Required Core Skills:

  • AWS
  • Python
  • PySpark
  • SQL
  • Shell scripting

Minimum years of experience: 7 years

Areas of responsibility / Detailed Job Description:

  • The successful candidate will work as a lead on-site, overseeing the development and implementation of data integration solutions across multiple platforms.
  • They will be responsible for setting up data connections from various sources, building and transforming data in SQL or PySpark/Python, and creating data pipelines using pipeline builders or code repositories.
  • The ideal candidate will have expertise in configuring and managing middleware agents on Linux, as well as debugging and optimizing failures in the Spark framework and Java.
  • They will also be responsible for administering the platform, including whitelisting endpoints, setting up ingress/egress policies, and managing SAML configurations.
  • Additionally, they will perform governance operations, such as identifying and rectifying PII data, setting up access approvals, and performing vulnerability fixes and resource management on servers.
  • The successful candidate will also support ontologies and other data pipelines, ensuring seamless data flow and integration across the organization.