Data Engineer

1 week ago

Newcastle upon Tyne, United Kingdom Version 1 Full time
Job Description

The Data Engineer will be implementing data ingestion and transformation pipelines for large scale organisations. We are seeking someone with deep technical skills in a variety of technologies to play an important role in developing and delivering early proofs of concept and production implementation.

You will be building solutions using a variety of open-source tools & Microsoft Azure services, and a proven track record in delivering high-quality work to tight deadlines.

Your main responsibilities will be:

  • Designing and implementing highly performant data ingestion & transformation pipelines from multiple sources using a variety of technologies
  • Delivering and presenting proofs of concept of key technology components to prospective customers and project stakeholders.
  • Developing scalable and re-usable frameworks for ingestion and transformation of large data sets
  • Master data management system and process design and implementation.
  • Data quality system and process design and implementation.
  • Integrating the end to end data pipeline to take data from source systems to target data repositories ensures the quality and consistency of data is maintained at all times
  • Working with event-based / streaming technologies to ingest and process data
  • Working with other members of the project team to support the delivery of additional project components (Reporting tools, API interfaces, Search)
  • Evaluating the performance and applicability of multiple tools against customer requirements
  • Working within an Agile delivery / DevOps methodology to deliver proof of concept and production implementation in iterative sprints.

Qualifications

  • Hands-on experience designing and delivering solutions using the Azure Data Analytics platform (Cortana Intelligence Platform) including Azure Storage, Azure SQL Database, Azure SQL Data Warehouse, Azure Data Lake, Azure Cosmos DB, Azure Stream Analytics
  • Direct experience in building data pipelines using Azure Data Factory and Apache Spark (preferably Databricks).
  • Experience building data warehouse solutions using ETL / ELT tools such as SQL Server Integration Services (SSIS), Oracle Data Integrator (ODI), Talend, and Wherescape Red.
  • Experience with Azure Event Hub, IOT Hub, Apache Kafka, Nifi for use with streaming data / event-based data
  • Experience with other Open Source big data products eg Hadoop (incl. Hive, Pig, Impala)
  • Experience with Open Source non-relational / NoSQL data repositories (incl. MongoDB, Cassandra, Neo4J)
  • Experience working with structured and unstructured data including imaging & geospatial data.
  • Comprehensive understanding of data management best practices including demonstrated experience with data profiling, sourcing, and cleansing routines utilizing typical data quality functions involving standardization, transformation, rationalization, linking and matching.
  • Experience working in a Dev/Ops environment with tools such as Microsoft Visual Studio Team Services, Chef, Puppet or Terraform


Additional Information

At Version 1, we believe in providing our employees with a comprehensive benefits package that prioritizes their well-being, professional growth, and financial stability.

One of our standout advantages is the ability to work remote or with a hybrid schedule along with business travel, allowing our employees to strike a balance between work and life. We also offer a range of tech-related benefits, including an innovative Tech Scheme, Octopus Car Lease, and MyGym Discounts to help keep our team members up-to-date with the latest technology.

We prioritize the health and safety of our employees, providing private medical and life insurance coverage, as well as free eye tests and contributions towards glasses. Our team members can also stay ahead of the curve with incentivized certifications and accreditations, including AWS, Microsoft, Oracle, and Red Hat.

#LI-LC2

  • Data Engineer

    2 weeks ago

    Newcastle Upon Tyne, United Kingdom Harnham Full time

    **DATA ENGINEER** **Durham 1x week** **Up to £50,000 per annum** **THE COMPANY** This company is one of the largest electrical wholesalers in the UK, which has now branched out to the USA, and is turning over $3 billion per year! They work with businesses and independent customers to provide the highest service and pride themselves ontheir fantastic...

  • Data Engineer

    1 week ago

    Newcastle upon Tyne, United Kingdom Adecco Full time

    Job Title: Data EngineerLocation: RemoteDuration: 6 monthsSalary: £104,000 per year Join our client's dynamic and innovative team as a Data Engineer! Are you ready to take the next step in your career as a Data Engineer? Do you have a passion for analytics and a knack for solving complex data challenges? If so, we have an exciting opportunity for you to...

  • Data Engineer

    1 month ago

    Newcastle upon Tyne, United Kingdom BJSS Full time

    About UsnWe’re an award-winning innovative tech consultancy - a team of creative problem solvers. Since 1993 we’ve been finding better, more sustainable ways to solve complex technology problems for some of the world’s leading organisations and delivered solutions that millions of people use every day.In the last 30 years we won several awards,...

  • Data Engineer

    3 weeks ago

    Newcastle upon Tyne, United Kingdom BJSS Full time

    About UsnWe’re an award-winning innovative tech consultancy - a team of creative problem solvers. Since 1993 we’ve been finding better, more sustainable ways to solve complex technology problems for some of the world’s leading organisations and delivered solutions that millions of people use every day.In the last 30 years we won several awards,...

  • Data Engineer

    4 weeks ago

    Newcastle upon Tyne, United Kingdom BJSS Full time

    Job Description About Us We’re an award-winning innovative tech consultancy - a team of creative problem solvers. Since 1993 we’ve been finding better, more sustainable ways to solve complex technology problems for some of the world’s leading organisations and delivered solutions that millions of people use every day. In the last 30 years we won...

  • Data Engineer

    1 week ago

    Newcastle upon Tyne, Newcastle upon Tyne, United Kingdom Steria Recruitment Full time

    Job Title: Senior Data EngineerLocation: Hybrid/NewcastleDuration: PermanentRole:As the Senior Data Engineer, you'll be joining the Digital Policing function and play a key role in providing effective data solutions but will equally understand the impact of a data failure to the business and support the user through the experience toits resolution.You will...

  • Data Engineer

    1 week ago

    Newcastle Upon Tyne, United Kingdom Oliver Bernard Full time

    Data Engineer - Very competitive Salary for the NEWe have partnered with a top London Hedge Fund & strong consultancy who are spinning up a joint venture in Newcastle. You will have a chance to shape the direction as the company grows from its infancy to around 20 heads over the next 6 months.They are looking for people with a strong education who can work...

  • Data Engineer

    1 week ago

    Newcastle Upon Tyne, United Kingdom Oliver Bernard Full time

    Data Engineer - Very competitive Salary for the NEWe have partnered with a top London Hedge Fund & strong consultancy who are spinning up a joint venture in Newcastle. You will have a chance to shape the direction as the company grows from its infancy to around 20 heads over the next 6 months.They are looking for people with a strong education who can work...

  • Data Engineer

    1 week ago

    Newcastle upon Tyne, United Kingdom Oliver Bernard Full time

    Data Engineer - Very competitive Salary for the NEWe have partnered with a top London Hedge Fund & strong consultancy who are spinning up a joint venture in Newcastle. You will have a chance to shape the direction as the company grows from its infancy to around 20 heads over the next 6 months.They are looking for people with a strong education who can work...

  • Data Engineer

    7 days ago

    Newcastle upon Tyne, Newcastle upon Tyne, United Kingdom Harnham Full time

    Data Engineer(This role does not offer sponsorship)Up to £55,000 per annumWFH:once a month - Leeds-based officeWe have a fantastic new opportunity for a data engineer at a large financial services company.The CompanyThis company is a global information and insights company. They offer specialist services in fraud, identity and risk management, automated...

  • Data Engineer

    1 week ago

    Newcastle upon Tyne, United Kingdom Oliver Bernard Full time

    Data Engineer - Very competitive Salary for the NEApplying for this role is straight forward Scroll down and click on Apply to be considered for this position.We have partnered with a top London Hedge Fund & strong consultancy who are spinning up a joint venture in Newcastle. You will have a chance to shape the direction as the company grows from its infancy...

  • Data Engineer

    4 weeks ago

    Newcastle upon Tyne, United Kingdom Xcede Full time

    Data Engineer (AWS)£70,000 Salary + BonusLondon office 2-3 days per week.OVERVIEWA successful AI scale up in the creative space are hiring for a Data Engineer (AWS) to join their well established Data team, spearheading some of their key Data Engineering projects in AWS and mentoring junior team members. Your responsibilities as a Data Engineer will include...

  • Lead Data Engineer

    1 month ago

    Newcastle upon Tyne, United Kingdom James Adams Full time

    About the Client: Our client are a tech for good business dedicated to promoting transparency in the corporate world they are seeking a highly skilled and experienced Lead Data Engineer. You will be the first hire in the Data Engineering team allowing you to take the drivers seat regarding the direction taken by the data team to follow.Key...

  • Data Engineer

    4 weeks ago

    Newcastle upon Tyne, United Kingdom DWP Digital Full time

    Digital with Purpose.Are you ready to join a dynamic team at the forefront of digital transformation?DWP Digital is seeking Data Engineers who are passionate about utilising cutting-edge technologies to deliver impactful data products. DWP is UK's largest department, every year we pay over £120 billion to 18million DWP customers.DWP services cover...

  • Junior Data Engineer

    3 weeks ago

    Newcastle Upon Tyne, United Kingdom Sparta Global Full time

    **Junior Data Engineer** **Government project - applicants must have a valid UK passport and a clean criminal record.** - A Junior Data Engineer will specialise in data and data technologies - connecting all pieces of the data ecosystem together within a company or institution. Become a Junior / Graduate Data Engineer today._ **What You'll do.** As a Data...

  • Data Engineer

    2 days ago

    Newcastle upon Tyne, United Kingdom DWP Digital Full time

    Digital with Purpose. DWP Digital is seeking a Data Engineer who is passionate about utilising cutting-edge technologies to deliver impactful data products. DWP is the UK's largest department, every year we pay over £195 billion to over 20 million citizens. DWP services cover everything from helping children & families right through to supporting...

  • Data Engineer

    4 weeks ago

    Newcastle upon Tyne, United Kingdom Version 1 Full time

    Version 1 has celebrated over 26 years in the Technology industry and continues to be trusted by global brands to deliver IT solutions that drive customer success. Version 1 has several strategic technology partners including Microsoft, AWS, Oracle, Red Hat, OutSystems and Snowflake. We’re also an award-winning employer reflecting how employees are at the...

  • Data Engineer

    3 weeks ago

    Newcastle upon Tyne, United Kingdom Version 1 Full time

    Version 1 has celebrated over 26 years in the Technology industry and continues to be trusted by global brands to deliver IT solutions that drive customer success. Version 1 has several strategic technology partners including Microsoft, AWS, Oracle, Red Hat, OutSystems and Snowflake. We’re also an award-winning employer reflecting how employees are at the...

  • Data Engineer

    1 month ago

    Newcastle upon Tyne, United Kingdom Version 1 Full time

    Version 1 has celebrated over 26 years in the Technology industry and continues to be trusted by global brands to deliver IT solutions that drive customer success. Version 1 has several strategic technology partners including Microsoft, AWS, Oracle, Red Hat, OutSystems and Snowflake. We’re also an award-winning employer reflecting how employees are at the...

  • Junior Data Engineer

    1 week ago

    Newcastle upon Tyne, Newcastle upon Tyne, United Kingdom Sparta Global Full time

    Junior Data EngineerGovernment project - applicants must have a valid UK passport and a clean criminal record. A Junior Data Engineer will specialise in data and data technologies connecting all pieces of the data ecosystem together within a company or institution. Become a Junior / Graduate Data Engineer today._What You'll do.As a Data Engineer, you will...