Data Pipeline Architect

6 days ago

London, Greater London, United Kingdom oilandgas Full time
About the Role

We are seeking a highly skilled Data Pipeline Engineer to join our team at Oil and Gas. As a Data Pipeline Engineer, you will play a critical role in designing, building, and maintaining efficient, reusable, and reliable data pipelines to support our business operations.

Key Responsibilities
  • Design and Development: Design and develop efficient data pipelines using Python to collect, process, and transform data from various sources.
  • Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand and fulfill their data requirements.
  • Performance Optimization: Ensure the performance, quality, and responsiveness of data pipelines to meet business needs.
  • Problem-Solving: Identify bottlenecks and bugs, and devise solutions to these problems.
  • Code Quality and Maintenance: Maintain high-quality code, organization, and automatization of data pipelines.
  • Integration: Collaborate with teams to integrate data pipelines with other systems.
Requirements
  • 4+ years of experience in a Data Pipeline Engineer role.
  • Strong programming skills in Python.
  • Experience with big data tools: Hadoop, Spark, Kafka, etc.
  • Experience with relational SQL and NoSQL databases.
  • Strong analytic skills related to working with unstructured datasets.
  • Strong project management and organizational skills.
  • Experience supporting and working with cross-functional teams in a dynamic environment.
Preferred Qualifications
  • Degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field.
  • Experience with machine learning algorithms and libraries.
  • Familiarity with data visualization tools.

  • Data Pipeline Architect

    3 weeks ago

    London, Greater London, United Kingdom Gismart Full time

    About Gismart Gismart is a driven mobile app development company with a strong presence in the Health & Wellness, Utilities, and Music app markets. We have achieved significant success, reaching over 1 billion downloads worldwide, as we work towards our mission of cultivating well-being globally. Headquartered in London, UK, Gismart is a dynamic global...

  • Data Pipeline Architect

    6 hours ago

    London, Greater London, United Kingdom Tether Operations Limited Full time

    About the RoleTether Operations Limited is seeking a highly skilled Data Pipeline Engineer to join our dynamic team. As a key member of our AI infrastructure team, you will play a crucial role in building the next generation of AI models and solutions.Key ResponsibilitiesDesign, develop, and maintain efficient data pipelines using Python and big data tools...

  • Data Pipeline Architect

    4 hours ago

    London, Greater London, United Kingdom Tether Operations Limited Full time

    About the RoleTether Operations Limited is seeking a highly skilled Data Pipeline Engineer to join our dynamic team. As a key member of our AI infrastructure team, you will play a crucial role in building the next generation of AI models and solutions.Key ResponsibilitiesDesign, develop, and maintain efficient data pipelines using Python and big data tools...

  • Cloud-Scale Data Pipeline Architect

    1 week ago

    London, Greater London, United Kingdom Understanding Recruitment Full time £90,000

    Senior Data Engineer - Geospatial & AIWe're seeking a talented Senior Data Engineer to join our innovative platform team in using advanced geospatial data and AI to tackle global challenges.Key Responsibilities:Design, develop, and deploy tools enabling solution engineers to create cloud-scale data pipelinesOwn critical data pipeline tools and best...

  • Cloud-Scale Data Pipeline Architect

    6 days ago

    London, Greater London, United Kingdom Understanding Recruitment Full time £90,000

    Senior Data Engineer - Geospatial & AIWe're seeking a talented Senior Data Engineer to join our innovative platform team in using advanced geospatial data and AI to tackle global challenges.Key Responsibilities:Design, develop, and deploy tools enabling solution engineers to create cloud-scale data pipelinesOwn critical data pipeline tools and best...

  • Data Pipeline Engineer

    1 week ago

    London, Greater London, United Kingdom Pearson Full time

    About the RoleThe Data Management & Analytics Analyst will play a crucial role in ensuring the integrity and assurance of data through multiple pipelines, as well as working towards better methods of exploiting this data in terms of analytics and reporting.Key ResponsibilitiesDesign and implement data pipelines to ensure data accuracy and integrityDevelop...

  • Data Pipeline Engineer

    7 days ago

    London, Greater London, United Kingdom Pearson Full time

    About the RoleThe Data Management & Analytics Analyst will play a crucial role in ensuring the integrity and assurance of data through multiple pipelines, as well as working towards better methods of exploiting this data in terms of analytics and reporting.Key ResponsibilitiesDesign and implement data pipelines to ensure data accuracy and integrityDevelop...

  • Data Pipeline Architect

    1 week ago

    London, Greater London, United Kingdom Tether Operations Limited Full time

    About Tether Operations LimitedTether Operations Limited is a pioneering company in the field of artificial intelligence, committed to making advanced AI technologies more accessible. Our investment in AI infrastructure has positioned us for tackling ambitious AI projects, and our goal is to build the next generation of AI models, leading innovation in AI...

  • Data Pipeline Architect

    6 days ago

    London, Greater London, United Kingdom Tether Operations Limited Full time

    About Tether Operations LimitedTether Operations Limited is a pioneering company in the field of artificial intelligence, committed to making advanced AI technologies more accessible. Our investment in AI infrastructure has positioned us for tackling ambitious AI projects, and our goal is to build the next generation of AI models, leading innovation in AI...

  • Data Architect

    1 week ago

    London, Greater London, United Kingdom Movement8 Full time

    Job OverviewMovement8 is seeking a highly skilled Data Architect to join our team as an Azure Data Engineer. This is a contract position, working on a financial services project that requires expertise in delivering and implementing new data pipelines.Key Responsibilities:Design and implement data pipelines using Azure Data Factory and DatabricksProvide...

  • Data Pipeline Architect

    1 week ago

    London, Greater London, United Kingdom DATAHEAD Full time £60,000 - £65,000

    About the RoleDATAHEAD is seeking a highly skilled Data Scientist to join our team in London. As a Data Scientist, you will play a key role in designing and implementing scalable data pipelines, ensuring data quality and integrity, and collaborating with cross-functional teams to develop end-to-end data solutions.Key ResponsibilitiesDesign and implement...

  • Data Pipeline Architect

    1 week ago

    London, Greater London, United Kingdom Tether Operations Limited Full time

    About Tether Operations LimitedWe are a pioneering company in the AI industry, committed to making advanced AI technologies more accessible. Our investment in AI infrastructure has positioned us for tackling ambitious AI projects, and our goal is to build the next generation of AI models through an accessible, transparent, and privacy-preserving approach.The...

  • Data Pipeline Architect

    6 days ago

    London, Greater London, United Kingdom Tether Operations Limited Full time

    About Tether Operations LimitedWe are a pioneering company in the AI industry, committed to making advanced AI technologies more accessible. Our investment in AI infrastructure has positioned us for tackling ambitious AI projects, and our goal is to build the next generation of AI models through an accessible, transparent, and privacy-preserving approach.The...

  • AWS Lead Data Engineer

    4 days ago

    London, Greater London, United Kingdom Fintop Full time

    Job DescriptionJob Title: AWS Lead Data EngineerCompany: FintopJob Type: PermanentAbout the Role:We are seeking an experienced AWS Lead Data Engineer to join our Engineering team within the Digital & Product Department at Fintop. As a key member of our team, you will be responsible for leading a team of data engineers in designing and creating scalable and...

  • AWS Lead Data Engineer

    4 days ago

    London, Greater London, United Kingdom Fintop Full time

    Job DescriptionJob Title: AWS Lead Data EngineerCompany: FintopJob Type: PermanentAbout the Role:We are seeking an experienced AWS Lead Data Engineer to join our Engineering team within the Digital & Product Department at Fintop. As a key member of our team, you will be responsible for leading a team of data engineers in designing and creating scalable and...

  • Data Pipeline Architect

    1 week ago

    London, Greater London, United Kingdom Aurum Search Limited Full time

    Job OpportunityAurum Search Limited is seeking a skilled Software Engineer to join our team in a Global Hedge Fund's London office. The ideal candidate will be responsible for developing high-quality applications and passionate about data and building efficient pipelines for data processing as part of an investment team.Key ResponsibilitiesDesign and...

  • Data Engineer

    1 week ago

    London, Greater London, United Kingdom Canada Life Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at Canada Life. As a Data Engineer, you will play a crucial part in driving data-driven decision-making across the organization, ensuring data availability, quality, and accessibility for various business needs.Key ResponsibilitiesData Pipeline DevelopmentDesign, model, develop, and...

  • Data Engineer

    6 days ago

    London, Greater London, United Kingdom Canada Life Full time

    Job SummaryCanada Life is seeking a highly skilled Data Engineer to join our team. As a Data Engineer, you will play a crucial part in designing, developing, and maintaining data pipelines that drive data-driven decision making across the organization.Key ResponsibilitiesData Pipeline DevelopmentDesign, model, develop, and maintain data pipelines to ingest,...

  • Data Engineer

    6 days ago

    London, Greater London, United Kingdom Canada Life Full time

    Job SummaryCanada Life is seeking a highly skilled Data Engineer to join our team. As a Data Engineer, you will play a crucial part in designing, developing, and maintaining data pipelines that drive data-driven decision making across the organization.Key ResponsibilitiesData Pipeline DevelopmentDesign, model, develop, and maintain data pipelines to ingest,...

  • Cloud Data Architect

    1 week ago

    London, Greater London, United Kingdom NTT DATA Full time

    Job Overview About the Role NTT DATA is seeking a seasoned Lead Data Solutions Architect to join our dynamic team. As a key member of our data architectural capability, you will be responsible for delivering cloud-based data solutions using best-in-class RDBMS, ETL/ELT, and Cloud platforms for blue-chip customers across various sectors. Key Responsibilities ...