Data Engineer

1 month ago

Edinburgh, Edinburgh, United Kingdom Smart Data Foundry Full time
About the Role

We are seeking a highly skilled Data Engineer to join our team at Smart Data Foundry. As a Data Engineer, you will play a critical role in developing and maintaining our data processing architecture, ensuring the quality of our data, and reducing manual effort involved in our work.

Key Responsibilities
  • Design and develop data processing architecture and pipelines to optimize data quality and reduce manual effort.
  • Develop and own data cleaning, quality, and validation processes to ensure data accuracy and integrity.
  • Create and explain overviews and diagrams of processes and infrastructure to technical, semi-technical, and non-technical users.
  • Collaborate with data scientists, product owners, data partners, and other internal and external customers to understand and deliver team commitments.
  • Develop and evaluate rapid prototypes to test features with end users and support tool selection.
  • Integrate data from multiple sources, including APIs, sftp, and manual file transfers.
  • Monitor system performance and contribute to infrastructure optimization, considering utility and cost.
  • Test and mentor less experienced software engineers and quality engineers.
  • Create documentation, operating procedures, and diagrams.
Requirements
  • Proven programming experience with Python in a Linux environment.
  • Good understanding of software development practices, including code reviews, unit testing, data validation, TDD, source management, continuous integration, and releasing software to live environments.
  • Knowledge of large-scale data processing technology, including Arrow, Hadoop, Cassandra, Spark, Hive, or Databricks.
  • Strong experience of batch ETL, data warehousing, database querying, and building secure and scalable applications in a cloud-based environment, preferably AWS.
Desirable Skills
  • Experience with infrastructure and container tools, such as Terraform, Kubernetes, and Docker/Podman.
  • Experience with modern software architecture paradigms, including microservices, event-driven, and serverless.
  • Experience with Jira Software or similar project management tools.
What We Offer

We offer a competitive reward package, 40 days annual leave, a defined contribution pension scheme, family-friendly initiatives, and flexible working. If you are a motivated and experienced Data Engineer looking for a new challenge, please apply.

  • Data Engineer

    4 weeks ago

    Edinburgh, Edinburgh, United Kingdom Smart Data Foundry Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at Smart Data Foundry. As a Data Engineer, you will play a critical role in developing and maintaining our data processing architecture, ensuring the quality of our data, and reducing manual effort involved in our work.Key ResponsibilitiesDesign and develop data processing...

  • Data Engineer

    4 weeks ago

    Edinburgh, Edinburgh, United Kingdom Smart Data Foundry Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at Smart Data Foundry. As a Data Engineer, you will play a critical role in developing and maintaining our data processing architecture, ensuring the quality of our data, and reducing manual effort involved in our work.Key ResponsibilitiesDesign and develop data processing...

  • Senior Data Engineer

    4 weeks ago

    Edinburgh, Edinburgh, United Kingdom Smart Data Foundry Full time

    {"title": "Data Engineer", "description": "About the RoleWe are seeking a highly skilled Data Engineer to join our team at Smart Data Foundry. As a Data Engineer, you will play a key role in designing and developing our data processing architecture and ensuring the quality of our data.Key ResponsibilitiesDevelop and own data cleaning, quality, and validation...

  • Senior Data Engineer

    4 weeks ago

    Edinburgh, Edinburgh, United Kingdom Smart Data Foundry Full time

    {"title": "Data Engineer", "description": "About the RoleWe are seeking a highly skilled Data Engineer to join our team at Smart Data Foundry. As a Data Engineer, you will play a key role in designing and developing our data processing architecture and ensuring the quality of our data.Key ResponsibilitiesDevelop and own data cleaning, quality, and validation...

  • Data Engineer

    1 month ago

    Edinburgh, Edinburgh, United Kingdom Home Technology Solutions Full time

    Unlock Your Potential as a Data EngineerAt Home Technology Solutions, we're on a mission to revolutionize the way we interact with technology. As a Data Engineer, you'll play a crucial role in shaping our digital future.About the RoleWe're seeking a talented Data Engineer to join our team and help us build innovative data-driven solutions. In this role,...

  • Data Engineer

    4 weeks ago

    Edinburgh, Edinburgh, United Kingdom MBN Solutions Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our Data and Analytics team at MBN Solutions. As a Data Engineer, you will play a key role in building and maintaining our data platform, working closely with stakeholders to develop and support new projects and deployments.Key ResponsibilitiesDesign, build, and maintain data pipelines, data...

  • Data Engineer

    4 weeks ago

    Edinburgh, Edinburgh, United Kingdom MBN Solutions Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our Data and Analytics team at MBN Solutions. As a Data Engineer, you will play a key role in building and maintaining our data platform, working closely with stakeholders to develop and support new projects and deployments.Key ResponsibilitiesDesign, build, and maintain data pipelines, data...

  • Data Engineer

    4 weeks ago

    Edinburgh, Edinburgh, United Kingdom MBN Solutions Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our Data and Analytics team at MBN Solutions. As a Data Engineer, you will play a key role in building and maintaining our data platform, working closely with stakeholders to develop and support new projects and deployments.Key ResponsibilitiesDesign, build, and maintain data pipelines, data...

  • Data Engineer

    4 weeks ago

    Edinburgh, Edinburgh, United Kingdom MBN Solutions Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our Data and Analytics team at MBN Solutions. As a Data Engineer, you will play a key role in building and maintaining our data platform, working closely with stakeholders to develop and support new projects and deployments.Key ResponsibilitiesDesign, build, and maintain data pipelines, data...

  • Data Engineer

    3 weeks ago

    Edinburgh, Edinburgh, United Kingdom Blend360 Full time

    Job DescriptionAt Blend360, we're seeking a talented Data Engineer to join our team. As a Data Engineer, you'll play a crucial role in driving high delivery standards and innovation within the company.Key Responsibilities:Design, develop, deploy, and maintain ingestion, transformation, and storage solutions.Analyze and organize raw data to deliver actionable...

  • Data Engineer

    3 weeks ago

    Edinburgh, Edinburgh, United Kingdom Blend360 Full time

    Job DescriptionAt Blend360, we're seeking a talented Data Engineer to join our team. As a Data Engineer, you'll play a crucial role in driving high delivery standards and innovation within the company.Key Responsibilities:Design, develop, deploy, and maintain ingestion, transformation, and storage solutions.Analyze and organize raw data to deliver actionable...

  • Data Engineer

    3 weeks ago

    Edinburgh, Edinburgh, United Kingdom Blend360 Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at Blend360. As a Data Engineer, you will play a critical role in driving high delivery standards and innovation in our company.Reporting to a Senior Data Engineer and working closely with the Data Science and Business Development teams, this role will be responsible for delivering...

  • Data Engineer

    3 weeks ago

    Edinburgh, Edinburgh, United Kingdom Blend360 Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at Blend360. As a Data Engineer, you will play a critical role in driving high delivery standards and innovation in our company.Reporting to a Senior Data Engineer and working closely with the Data Science and Business Development teams, this role will be responsible for delivering...

  • Data Engineer

    6 days ago

    Edinburgh, Edinburgh, United Kingdom Blend360 Full time

    Job Title: Data EngineerAbout the Role:We are seeking a highly skilled Data Engineer to join our team at Blend360. As a Data Engineer, you will be responsible for designing, developing, and deploying data solutions to support the provision of actionable insights for stakeholders.Key Responsibilities:Preparing and presenting data-driven solutions to...

  • Data Engineer

    1 week ago

    Edinburgh, Edinburgh, United Kingdom Blend360 Full time

    Job Title: Data EngineerAbout the Role:We are seeking a skilled Data Engineer to join our team at Blend360. As a Data Engineer, you will be responsible for designing, developing, and deploying data solutions to support the provision of actionable insights for stakeholders.Key Responsibilities:Preparing and presenting data-driven solutions to...

  • Data Engineer

    1 month ago

    Edinburgh, Edinburgh, United Kingdom BJSS Full time

    About UsBJSS is a leading tech consultancy that has been delivering innovative solutions to clients for over 30 years. Our team of creative problem solvers is committed to finding better, more sustainable ways to solve complex technology problems.We have a strong track record of delivering award-winning solutions that have a significant impact on our...

  • Data Engineer

    1 month ago

    Edinburgh, Edinburgh, United Kingdom BJSS Full time

    About UsBJSS is a leading tech consultancy that has been delivering innovative solutions to clients for over 30 years. Our team of creative problem solvers is committed to finding better, more sustainable ways to solve complex technology problems.We have a strong track record of delivering award-winning solutions that have a significant impact on our...

  • Data Engineer

    1 month ago

    Edinburgh, Edinburgh, United Kingdom MBN Solutions Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our Data and Analytics team. As a Data Engineer, you will be responsible for designing, building, and maintaining large-scale data systems, ensuring data quality, and developing data pipelines to support business decisions.Key ResponsibilitiesDesign and implement data pipelines using Azure...

  • Data Engineer

    1 month ago

    Edinburgh, Edinburgh, United Kingdom BJSS Full time

    About the RoleAt BJSS, we're seeking a skilled Data Engineer to join our team of specialist software engineers. As a Data Engineer, you'll play a crucial role in building, optimising, and maintaining data applications, systems, and services that deliver business value.Our Data Engineers work in a fast-paced, agile environment, collaborating with...

  • Data Engineer

    1 month ago

    Edinburgh, Edinburgh, United Kingdom BJSS Full time

    About the RoleAt BJSS, we're seeking a skilled Data Engineer to join our team of specialist software engineers. As a Data Engineer, you'll play a crucial role in building, optimising, and maintaining data applications, systems, and services that deliver business value.Our Data Engineers work in a fast-paced, agile environment, collaborating with...