Data Engineer

2 days ago

Edinburgh, Edinburgh, United Kingdom Athsai Full time

Data Engineer AWS Pyspark SQL Redshift Lambda

Scotland Remote

**About the Role**

The Data Engineer will play a pivotal role in organization by designing and implementing robust data pipelines that facilitate efficient data flow and management across various platforms. This position is essential for ensuring the integrity, reliability, and accessibility of our data, which supports critical business decisions and drives insights.

**Required Skills**

  • **Proficiency in PySpark and AWS:** You should have a strong command of both PySpark for data processing and AWS (Amazon Web Services) for cloud-based solutions.

  • **ETL Pipeline Development:** Demonstrated experience in designing, implementing, and debugging ETL (Extract, Transform, Load) pipelines is crucial. You will be responsible for moving and transforming data from various sources to data warehouses.

  • **Programming Expertise:** A solid understanding of Python, PySpark, and SQL is required to manipulate and analyze data efficiently.

  • **Knowledge of Spark and Airflow:** In-depth knowledge of Apache Spark for big data processing and Apache Airflow for orchestrating complex workflows is essential for managing data pipelines.

  • **Cloud-Native Services:** Experience in designing data pipelines leveraging cloud-native services on AWS to ensure scalability and reliability in data handling.

  • **AWS Services:** Extensive knowledge of various AWS services, such as S3, RDS, Redshift, and Lambda, will be necessary for building and managing our data infrastructure.

  • **Terraform for Deployment:** Proficient in deploying AWS resources using Terraform, ensuring that infrastructure as code is implemented effectively.

  • **CI/CD Workflows:** Hands-on experience in setting up Continuous Integration and Continuous Deployment (CI/CD) workflows using GitHub Actions to automate the deployment process and enhance collaboration within the team.

**Preferred Skills**

  • **Data Governance and Compliance:** Understanding of data governance frameworks and compliance standards will be beneficial as we prioritize data privacy and regulatory requirements.

We are looking for a proactive and detail-oriented Data Engineer who is passionate about working with data and driving innovation . If you have a strong technical background and a commitment to excellence, we would love to hear from you

  • Data Operations Engineer

    1 week ago

    Edinburgh, Edinburgh, United Kingdom Smart Data Foundry Full time £35,000 - £55,000 per year

    This role is for 12-month maternity coverAbout Smart Data FoundrySmart Data Foundry has a purpose to unlock the power of financial data as a force to improve people's lives.What we doWe enable the research ecosystem to flourish through the provision of research ready real financial data.We create data-driven insights based on real financial data, that...

  • Senior Data Scientist

    1 week ago

    Edinburgh, Edinburgh, United Kingdom Smart Data Foundry Full time £50,000 - £90,000 per year

    About Smart Data FoundrySmart Data Foundry has a purpose to unlock the power of financial data as a force to improve people's lives.What we doWe enable the research ecosystem to flourish through the provision of research ready real financial data.We create data-driven insights based on real financial data, that identify areas to inform policy change and...

  • Data Scientist

    2 weeks ago

    Edinburgh, Edinburgh, United Kingdom AI Connect | Data & AI Delivery Partner Full time £45,000 - £55,000 per year

    Data ScientistEdinburgh City Centre (2 days per week in office)£45,000 - £50,000 + excellent packageAI Connect are proud to be partnered with a growing and influential energy trading organisation in Edinburgh, who use advanced data science and machine learning to tackle real-world energy challenges.This is a brilliant opportunity to join a team at the...

  • Data Engineer

    4 days ago

    Edinburgh, Edinburgh, United Kingdom Change Digital – Digital & Tech Recruitment Full time £40,000 - £80,000 per year

    Are you an experienced SAS Developer / Engineer ?Do you have experience with Cloud technologies ?Would you like to work for a fast growing business that specialises in Business Intelligence & Data Analytics ?Its a hybrid working role and you will be based from either their London, Leeds, Manchester or Edinburgh office, typically 3 days a week from home and 2...

  • Data Engineer

    2 weeks ago

    Edinburgh, Edinburgh, United Kingdom Leonardo Full time £60,000 - £90,000 per year

    Job DescriptionYour ImpactAs a Data Engineer, you will design, develop, deploy, and maintain data architecture which employs various methods to transform raw data into processed data. You will own the data operations infrastructure, manage and optimise performance, reliability, and scalability of the system to meeting growing demands on ingestion and...

  • Data Engineer

    1 week ago

    Edinburgh, Edinburgh, United Kingdom Leonardo Full time £75,000 - £110,000 per year

    Job Description:Your ImpactAs a Data Engineer, you will design, develop, deploy, and maintain data architecture which employs various methods to transform raw data into processed data.  You will own the data operations infrastructure, manage and optimise performance, reliability, and scalability of the system to meeting growing demands on ingestion and...

  • Data Engineering Manager

    1 week ago

    Edinburgh, Edinburgh, United Kingdom Cathcart Technology Full time £60,000 - £120,000 per year

    Lead a high-performing data engineering team in Edinburgh, driving the development of large-scale, reliable data systems that enable analytics, insight, and operational decision-making across a fast-paced technology organisation.The OpportunityYou'll oversee a team of skilled data engineers, guiding the design and delivery of scalable data pipelines and...

  • Data Engineer

    2 weeks ago

    Edinburgh, Edinburgh, United Kingdom GoFibre Full time £46,800 - £56,000

    Data EngineerEdinburgh Hybrid£46,800 - £56,000 31 days holiday, discounted gym membership, enhanced pension, private healthcare, employee wellbeing support and career coachingWHO WE AREAt GoFibre we're on an exciting journey to revolutionise broadband capabilities for homes and businesses in rural towns and villages across Scotland and the north of...

  • Lead Data Engineer

    2 weeks ago

    Edinburgh, Edinburgh, United Kingdom Royal London Group Full time £40,000 - £60,000 per year

    Job Title: Lead Data EngineerContract Type: PermanentLocation: Edinburgh or Alderley ParkWorking style: Hybrid 50% home/office basedClosing Date: 14th December 2025We are seeking to hire a Lead Data Engineer to design, develop and operate automated data engineering platforms.This role will contribute to the Group by improving the speed, accuracy,...

  • Lead Data Engineer

    1 week ago

    Edinburgh, Edinburgh, United Kingdom Royal London Full time £75,000 - £85,000 per year

    Job Title: Lead Data EngineerContract Type: Permanent Location: Edinburgh or Alderley ParkWorking style: Hybrid 50% home/office basedClosing Date: 14th December 2025We are seeking to hire a Lead Data Engineer to design, develop and operate automated data engineering platforms.This role will contribute to the Group by improving the speed, accuracy,...