Data Engineer

2 weeks ago

London, Greater London, United Kingdom Mastek Full time
Job Description

Job Description: Data Engineer

Location: [UK]

SC Cleared: Required

Job Type: Full-Time

Experience: 12-14 years

Job description:

We are seeking a highly skilled and motivated Data Engineer to join our dynamic team. The Data Engineer will be responsible for designing, building, and maintaining scalable data pipelines and infrastructure to support data-driven initiatives. The successful candidate will work closely with cross-functional teams to ensure efficient data processing, storage, and analysis.

Key Responsibilities:

Data Pipeline Development:

Design, develop, and maintain scalable data pipelines to ingest, process, and transform large volumes of data.

  • Implement data integration solutions to ensure seamless data flow across systems.
  • Optimize data pipelines for performance, reliability, and scalability.
  • Data Infrastructure Management: Build and maintain data storage and processing infrastructure, including databases, data warehouses, and data lakes.
  • Configure and optimize database systems for performance and reliability.
  • Implement data partitioning, indexing, and optimization strategies.
  • Data Modeling and Schema Design: Design and implement data models and schemas to support business requirements.
  • Ensure data models are efficient, scalable, and maintainable.
  • Collaborate with data architects and analysts to design data structures for analytics and reporting.
  • Data Quality and Governance: Implement data quality checks and validation processes to ensure data accuracy and consistency.
  • Establish data governance policies and procedures to ensure compliance with regulatory requirements and industry standards.
  • Monitor data quality metrics and implement corrective actions as needed.
  • Tools and Technologies: Evaluate and recommend data engineering tools and technologies.
  • Stay up-to-date with emerging trends and technologies in data engineering and analytics.
  • Collaborate with IT teams to implement and integrate data engineering solutions.
  • Documentation and Collaboration: Document data pipelines, workflows, and infrastructure configurations.
  • Collaborate with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand data requirements and priorities.
  • Qualifications: Bachelor's or Master's degree in Computer Science, Information Technology, or related field.
  • 3+ years of experience in data engineering, ETL development, or related roles.
  • Proficiency in programming languages such as Python, Java, or Scala.
  • Experience with data processing frameworks such as Apache Spark, Apache Flink, or Apache Beam.
  • Strong understanding of database technologies (e.g., SQL, NoSQL) and data warehousing concepts.
  • Familiarity with cloud platforms such as AWS, Azure, or Google Cloud Platform.
  • Excellent analytical, problem-solving, and communication skills.
  • Ability to work independently and collaboratively in a fast-paced environment.
  • Key Competencies: Data pipeline development and optimization
  • Data infrastructure management
  • Data modeling and schema design
  • Data quality and governance
  • Tools and technologies proficiency
  • Documentation and collaboration skills

  • Data Engineer

    2 weeks ago

    London, Greater London, United Kingdom Data Idols Full time

    Data Engineer Are you a mid-level Data Engineer skilled in Azure technologies, seeking an new 'feel-good' permanent opportunity? Data Idols are proud to be supporting a HealthTech client, who use remote-first technology focusing on engagingpatients across their healthcare journey while saving hospitals time, effort, and money This opportunity provides remote...

  • Senior Engineer, Data Engineering

    2 weeks ago

    London, Greater London, United Kingdom Data Idols Full time £85,000

    Job Description Senior Data Engineer Location: London (Hybrid) Data Idols are working with an exciting scale-up revolutionising delivery logistics and moving items across the UK with efficiency and ease all whilst cutting emissions. Trusted by millions of customers already they are expanding their data team to manage increased volumes of data and demand. As...

  • Senior Data Engineer

    2 weeks ago

    London, Greater London, United Kingdom Data Idols Full time

    Senior Data EngineerIncrease your chances of reaching the interview stage by reading the complete job description and applying promptly.Salary: up to £85,000Location: London (Hybrid)Data Idols are working with an exciting scale-up revolutionising delivery logistics and moving items across the UK with efficiency and ease all whilst cutting emissions. Trusted...

  • NS - Data Engineer

    2 weeks ago

    London, Greater London, United Kingdom Abs Data Full time

    Job title: NS – Data Engineer Company: BAE Systems Job description : Location(s): [[mfield3]] BAE Systems Digital Intelligence is home to 4,500 digital, cyber and intelligence experts. We work collaboratively across 10 countries to collect, connect and understand complex data, so that governments, nation states, armed forces and commercial businesses...

  • Senior Engineer, Data Engineering

    2 weeks ago

    London, Greater London, United Kingdom ON Data Staffing Full time

    Our client, a Data & AI firm in London is actively seeking a Senior Data Engineer to play a pivotal role in delivering innovative big data solutions to their end clients. This client-facing role requires a unique blend of technical prowess in Databricks/Snowflake and exceptional communication skills to understand and address clients' diverse data...

  • Senior Engineer, Data Engineering

    2 weeks ago

    London, Greater London, United Kingdom Data Science Festival Full time £85,000

    Senior Data Engineer Location: London (Hybrid) Data Idols are working with an exciting scale-up revolutionising delivery logistics and moving items across the UK with efficiency and ease all whilst cutting emissions. Trusted by millions of customers already they are expanding their data team to manage increased volumes of data and demand. As a Senior Data...

  • Senior Engineer, Data Engineering

    2 weeks ago

    London, Greater London, United Kingdom Data Science Festival Full time £70,000 - £85,000

    Senior Data Engineer Location: London, 3/2 days in the office per week Data Idols are currently working with a company that focus on the european marketplace for transport who are looking to make the future greener Within this role as a Senior Data Engineer, you will be monitoring, designing and implementing a datawarehouse, whilst working with different...

  • Senior Data Engineer

    2 weeks ago

    London, Greater London, United Kingdom Data Science Festival Full time £85,000

    Senior Data Engineer Salary: up to £85,000 Location: London (Hybrid) Data Idols are working with an exciting scale-up revolutionising delivery logistics and moving items across the UK with efficiency and ease all whilst cutting emissions. Trusted by millions of customers already they are expanding their data team to manage increased volumes of data and...

  • Data Engineer

    2 weeks ago

    London, Greater London, United Kingdom DATA MART Consulting GmbH Full time

    Your Role: Apollo is establishing a set of truly innovative data services to propel the business to the next level. We are capturing, transforming, and analysing a vast variety of data through a modern data platform to produce fabulously rich insight for our business. You are the enthusiastic, skilful, and creative Data Engineer we need to make this...

  • Data Engineering Manager

    2 weeks ago

    London, Greater London, United Kingdom ON Data Staffing Full time

    We are seeking a dynamic and experienced Data Engnieering professional to join our client, a growing consultancy, as a Data Engineering Manager.As Data Engineering Manager, you will play a pivotal role in driving the success of our client's Data Engineering practice through your expertise in presales, strong technical acumen, consulting skills, and effective...

  • Data Engineer

    2 weeks ago

    London, Greater London, United Kingdom Open Data Science Conferenc Full time

    You have prior experience as a data engineer, software engineer, or data architect. You are experienced in designing and production of solutions for complex problems. You thrive in a fast-paced, start-up environment. You seek to continuously improve processes and find the most optimal way to solve problems. You can work remotely and independently,...

  • Lead Data Engineer

    2 weeks ago

    London, Greater London, United Kingdom Harnham - Data & Analytics Recruitment Full time

    Lead Data Engineer - Residential Property CompanyLocation: London (flexible working available)Salary: CompetitiveWe are a leading residential property company looking to hire a Lead Data Engineer to join our growing data-focused team. You will be responsible for leading a team of data engineers and playing a key role in the development and maintenance of our...

  • Lead Data Engineer

    2 weeks ago

    London, Greater London, United Kingdom Harnham - Data & Analytics Recruitment Full time

    Lead Data Engineer Location: LondonSalary: £100,000 + BenefitsExciting Opportunity in LondonDo you want to work in the media sector for a top company? We are currently seeking a skilled Lead Data Engineer ready for a new challenge and eager to grow in their career.About the CompanyThis innovative media company is data-driven and is seeking a Lead Data...

  • Lead Data Engineer

    2 weeks ago

    London, Greater London, United Kingdom Harnham - Data & Analytics Recruitment Full time

    LEAD DATA ENGINEERLONDON, CURRENTLY FLEXIBLE / REMOTE / HOMEWORKINGUP TO £100,000 + BENEFITSAs a Lead Data Engineer, you will be inheriting a Snowflake data platform for a successful Ecommerce company that has disrupted the market with their product line. You will be leading a junior squad so will remain hands-on to further develop the platform and team...

  • Aws Data Engineer

    2 weeks ago

    London, Greater London, United Kingdom Harnham - Data & Analytics Recruitment Full time

    AWS Data Engineer£ a day, Inside IR35.Hybrid role, 2 days a week on-site. London office.6-Month Contract, potential to keep rollingFantastic opportunity to join an oil and gas company is an expanding team. You will join the business as an AWS Data Engineer playing a major role in a migration project that is ongoing, helping push it through to completion....

  • gcp data engineer

    2 weeks ago

    London, Greater London, United Kingdom Harnham - Data & Analytics Recruitment Full time

    GCP DATA ENGINEER, 6-MONTH CONTRACT LONDON (FULLY REMOTE) £600-£700 PER DAY This position as a GCP Engineer allows you to work within a dynamic Telecoms company located in the heart of London. You will be developing a new GCP data environment and migrating data into the new environment. THE COMPANY This Telecoms company has been very well established...

  • Gcp Data Engineer

    2 weeks ago

    London, Greater London, United Kingdom Harnham - Data & Analytics Recruitment Full time

    GCP Data Engineer£ a day, Inside IR35.Hybrid role, 2 days a week on-site. London office.6-Month Contract, potential to keep rollingFantastic opportunity to join an energy company is an expanding team. You will join the business as a GCP Data Engineer playing a major role in a migration project that is about to kick-off. This migration is from their legacy...

  • gcp data engineer

    2 weeks ago

    London, Greater London, United Kingdom Harnham - Data & Analytics Recruitment Full time

    GCP DATA ENGINEER, 6-MONTH CONTRACT LONDON (FULLY REMOTE) £600-£700 PER DAY This position as a GCP Engineer allows you to work within a dynamic Telecoms company located in the heart of London. You will be developing a new GCP data environment and migrating data into the new environment. THE COMPANY This Telecoms company has been very well established...

  • Senior Engineer, Data Engineering

    2 weeks ago

    London, Greater London, United Kingdom Open Data Science Conferenc Full time

    You have prior experience as a data engineer, software engineer, or data architect. You can work remotely and independently, taking ownership of your work. You have proven English language communication skills that enable you to collaborate in a remote-first company with a diverse international workforce. Experience with Java11+ for ingesting,...

  • Data Developer

    2 weeks ago

    London, Greater London, United Kingdom Data Intellect Full time

    Welcome to Data Intellect . Providing technical leadership on client projects Leading technical teams to deliver projects on time and on budget Developing, testing, supporting and maintaining all code for client applications Analysing data and creating reports for clients, incorporating all findings Mentorship, training and potentially management of...