Data Engineer

2 weeks ago


Newcastle upon Tyne, Newcastle upon Tyne, United Kingdom Version 1 Full time
Job Description

The Data Engineer will be implementing data ingestion and transformation pipelines for large scale organisations. We are seeking someone with deep technical skills in a variety of technologies to play an important role in developing and delivering early proofs of concept and production implementation.

You will be building solutions using a variety of open-source tools & Microsoft Azure services, and a proven track record in delivering high-quality work to tight deadlines.

Your main responsibilities will be:

  • Designing and implementing highly performant data ingestion & transformation pipelines from multiple sources using a variety of technologies
  • Delivering and presenting proofs of concept of key technology components to prospective customers and project stakeholders.
  • Developing scalable and re-usable frameworks for ingestion and transformation of large data sets
  • Master data management system and process design and implementation.
  • Data quality system and process design and implementation.
  • Integrating the end to end data pipeline to take data from source systems to target data repositories ensures the quality and consistency of data is maintained at all times
  • Working with event-based / streaming technologies to ingest and process data
  • Working with other members of the project team to support the delivery of additional project components (Reporting tools, API interfaces, Search)
  • Evaluating the performance and applicability of multiple tools against customer requirements
  • Working within an Agile delivery / DevOps methodology to deliver proof of concept and production implementation in iterative sprints.
Qualifications
  • Hands-on experience designing and delivering solutions using the Azure Data Analytics platform (Cortana Intelligence Platform) including Azure Storage, Azure SQL Database, Azure SQL Data Warehouse, Azure Data Lake, Azure Cosmos DB, Azure Stream Analytics
  • Direct experience in building data pipelines using Azure Data Factory and Apache Spark (preferably Databricks).
  • Experience building data warehouse solutions using ETL / ELT tools such as SQL Server Integration Services (SSIS), Oracle Data Integrator (ODI), Talend, and Wherescape Red.
  • Experience with Azure Event Hub, IOT Hub, Apache Kafka, Nifi for use with streaming data / event-based data
  • Experience with other Open Source big data products eg Hadoop (incl. Hive, Pig, Impala)
  • Experience with Open Source non-relational / NoSQL data repositories (incl. MongoDB, Cassandra, Neo4J)
  • Experience working with structured and unstructured data including imaging & geospatial data.
  • Comprehensive understanding of data management best practices including demonstrated experience with data profiling, sourcing, and cleansing routines utilizing typical data quality functions involving standardization, transformation, rationalization, linking and matching.
  • Experience working in a Dev/Ops environment with tools such as Microsoft Visual Studio Team Services, Chef, Puppet or Terraform
Additional Information

At Version 1, we believe in providing our employees with a comprehensive benefits package that prioritizes their well-being, professional growth, and financial stability.

One of our standout advantages is the ability to work remote or with a hybrid schedule along with business travel, allowing our employees to strike a balance between work and life. We also offer a range of tech-related benefits, including an innovative Tech Scheme, Octopus Car Lease, and MyGym Discounts to help keep our team members up-to-date with the latest technology.

We prioritize the health and safety of our employees, providing private medical and life insurance coverage, as well as free eye tests and contributions towards glasses. Our team members can also stay ahead of the curve with incentivized certifications and accreditations, including AWS, Microsoft, Oracle, and Red Hat.


  • Data Engineer

    2 weeks ago


    Newcastle upon Tyne, Newcastle upon Tyne, United Kingdom Steria Recruitment Full time

    Job Title: Senior Data EngineerLocation: Hybrid/NewcastleDuration: PermanentRole:As the Senior Data Engineer, you'll be joining the Digital Policing function and play a key role in providing effective data solutions but will equally understand the impact of a data failure to the business and support the user through the experience toits resolution.You will...

  • Data Engineer

    2 weeks ago


    Newcastle upon Tyne, Newcastle upon Tyne, United Kingdom Harnham Full time

    Data Engineer(This role does not offer sponsorship)Up to £55,000 per annumWFH:once a month - Leeds-based officeWe have a fantastic new opportunity for a data engineer at a large financial services company.The CompanyThis company is a global information and insights company. They offer specialist services in fraud, identity and risk management, automated...

  • Junior Data Engineer

    2 weeks ago


    Newcastle upon Tyne, Newcastle upon Tyne, United Kingdom Sparta Global Full time

    Junior Data EngineerGovernment project - applicants must have a valid UK passport and a clean criminal record. A Junior Data Engineer will specialise in data and data technologies connecting all pieces of the data ecosystem together within a company or institution. Become a Junior / Graduate Data Engineer today._What You'll do.As a Data Engineer, you will...


  • Newcastle upon Tyne, Newcastle upon Tyne, United Kingdom Oliver Bernard Full time

    Job Description Data Engineer - Very competitive Salary for the NE The role is truly unique to the North East and will require 5 days a week onsite in the centre of Newcastle. Data Engineer - Very competitive Salary for the NE

  • Lead Data Engineer

    2 weeks ago


    Newcastle upon Tyne, Newcastle upon Tyne, United Kingdom Reed Full time

    This is a true opportunity to step into a lead position, to elevate your career in an enterprise environment. These kinds of roles don't come around every day and this is a perfect springboard opportunity, for an experience senior or lead engineer, withsome people management experience.The VisionThe organisation is investing substantially in their Data...


  • Newcastle upon Tyne, Newcastle upon Tyne, United Kingdom Oliver Bernard Full time

    Graduate Data EngineerLocation: Newcastle, NE1 1RQWorking Arrangements: Hybrid, 3-4 days p/w in officeSalary: GBP40,000 - GBP50,000Industry: FinanceTech Stack: Python, SQL, SparkGreat opportunity for a Graduate Engineer (Python, SQL, Spark) to join a startup incubator within a world leading finance business. The Company Newly formed area within one of the...


  • Newcastle upon Tyne, Newcastle upon Tyne, United Kingdom BBC Full time

    Job BandPackage description Band: D Contract Permanent Salary : £65,000 to £85,000 depending on level of experience Location: Salford, and Newcastle Our comprehensive benefits package includes: An employer pension contribution of up to 10% 26 days' annual leave (based on full time hours) + bank holidays and the option to buy/sell additional days...


  • Newcastle upon Tyne, Newcastle upon Tyne, United Kingdom BBC Full time

    Job BandPackage description Band: D Contract Permanent Salary : £65,000 to £85,000 depending on level of experience Location: Salford, and Newcastle Our comprehensive benefits package includes: An employer pension contribution of up to 10% 26 days' annual leave (based on full time hours) + bank holidays and the option to buy/sell additional days...

  • Lead Data Engineer

    2 weeks ago


    Newcastle upon Tyne, Newcastle upon Tyne, United Kingdom Thoughtworks Inc. Full time

    Lead data engineers at Thoughtworks develop modern data architecture approaches to meet key business objectives and provide end-to-end data solutions. They might spend a few weeks with a new client on a deep technical review or a complete organizational review, helping them to understand the potential that data brings to solve their most pressing problems....

  • Senior Data Engineer

    4 weeks ago


    Newcastle upon Tyne, Newcastle upon Tyne, United Kingdom BBC Full time

    Job BandPackage description Band: DContract type: permanent / fixed term contractLocation: Newcastle, and SalfordSalary: £48,000 to £64,000 depending on experience Our comprehensive benefits package includes: An employer pension contribution of up to 10% 26 days' annual leave (based on full time hours) + bank holidays and the option to buy/sell...

  • Senior Data Engineer

    1 month ago


    Newcastle upon Tyne, Newcastle upon Tyne, United Kingdom BBC Full time

    Job BandPackage description Band: DContract type: permanent / fixed term contractLocation: Newcastle, and SalfordSalary: £48,000 to £64,000 depending on experience Our comprehensive benefits package includes: An employer pension contribution of up to 10% 26 days' annual leave (based on full time hours) + bank holidays and the option to buy/sell...

  • Senior Data Engineer

    2 weeks ago


    Newcastle upon Tyne, Newcastle upon Tyne, United Kingdom Version 1 Full time

    Version 1 has celebrated over 26 years in Technology Services and continues to be trusted by global brands to deliver solutions that drive customer success. Version 1 has several strategic technology partners including Microsoft, AWS, Oracle, Red Hat, OutSystems and Snowflake. We're also an award-winning employer reflecting how employees are at the heart of...

  • Principal Electrical

    2 weeks ago


    Newcastle upon Tyne, Newcastle upon Tyne, United Kingdom Abs Data Full time

    Job title: Principal Electrical / ICA Engineer – Newcastle Company: WSP Job description : We are WSP – Join us and make your career future ready In today's world it's important to work for a company that has clear purpose, giving back to communities and supporting what is truly important in the world. When considering a career move it's vital to...


  • Newcastle upon Tyne, Newcastle upon Tyne, United Kingdom BJSS Full time

    About the Role In Data at BJSS you'll join multi-disciplinary teams of some of the brightest and best in the industry. You'll work in a fast moving, agile environment building data models and ontologies, and analysing and profiling datasets. You'll design ETL processes and services and create powerful dashboards to bring data insights to life.You'll help us...


  • Newcastle upon Tyne, Newcastle upon Tyne, United Kingdom Reed Talent Solutions Full time

    If complex engineering projects, an employer that supports your development and recognises your achievements and a supportive and collaborative team environment are important to you then Black & White Engineering are an employer to consider. Your focus will be on designing electrical systems and solutions for complex and challenging global datacentre...

  • Data Analyst

    2 weeks ago


    Newcastle upon Tyne, Newcastle upon Tyne, United Kingdom EDWARD REED RECRUITMENT LIMITED Full time

    Edward Reed are seeking a highly motivated Business Analyst with strong technical, interpersonal, and business skills to make a significant contribution to our client. The Data Analyst will be responsible for specific processes within the ERP systems ofthe business, based in the North East.Main Duties Based on analysis of the problem, propose improvements to...

  • Data Migration Lead

    2 weeks ago


    Newcastle upon Tyne, Newcastle upon Tyne, United Kingdom Xplor Full time

    Company DescriptionTake a seat on the Xplor rocket ship and join us as Data Migration Lead to help people succeed across the world.We're a global team of builders, listeners and problem-solvers who are relentlessly focused on making life simple, so our customers can get back to growing their business, engaging consumers and doing what they love.Job...

  • Data Platform Lead

    2 weeks ago


    Newcastle upon Tyne, Newcastle upon Tyne, United Kingdom Greggs Full time

    About the role This is a fulltime role, however flexibility in this will be considered We know that having a worklife balance is important, so we offer our colleagues as much flexibility as possible in line with the needs of their role The base location for this role is Greggs Innovate House, Newcastle upon Tyne, NE12 8EWWhat you'll doReporting to the BI...


  • Newcastle upon Tyne, Newcastle upon Tyne, United Kingdom Northumbria University Full time

    We are looking for a Sea Ice Data Scientist to join the team at the Centre for Polar Observation and Modelling (CPOM) in the UK. CPOM has been a key player in providing satellite observations and model simulations of Earth's polar regions for over 25 years. As a Sea Ice Data Scientist, you will play a vital role in assessing the impacts of climate change on...

  • AI Data Scientist

    6 days ago


    Newcastle upon Tyne, Newcastle upon Tyne, United Kingdom Northumbria University Full time

    ABOUT THE ROLE Northumbria University in partnership with ROSEN(UK) Ltd are seeking a high-calibre graduate with an MSc (or equivalent) or other Higher Degree in a relevant field such Computer Science or Data Science with a focus on Artificial Intelligence and Machine Learning. You will take a leading role in an ambitious project, developing and testing...