Senior Data Engineer

2 weeks ago


Northumberland, United Kingdom Tech4 Full time

Senior Data Engineer - Python / Data Pipelines / Data Platform / AWS - is required by fast growing, highly successful and and tech focused organisation.

About the job

You will play a crucial role in designing, building, and maintaining their data platform, with a strong emphasis on streaming data, cloud infrastructure, and machine learning operations.

Key Responsibilities:

  • Architect and Implement Data Pipelines:
  • Design, develop, and maintain scalable and efficient data pipelines
  • Optimize ETL processes to ensure seamless data ingestion, processing, and integration across various systems
  • Streaming Data Platform Development:
  • Lead the development and maintenance of a real-time data streaming platform using tools like Apache Kafka, Databricks, Kinesis.
  • Ensure the integration of streaming data with batch processing systems for comprehensive data management
  • Cloud Infrastructure Management:
  • Utilize AWS data engineering services (including S3, Redshift, Glue, Kinesis, Lambda, etc.) to build and manage our data infrastructure
  • Continuously optimize the platform for performance, scalability, and cost-effectiveness
  • Communications:
  • Collaborate with cross-functional teams, including data scientists and BI developers, to understand data needs and deliver solutions
  • Leverage the project management team to coordinate project, requirements, timelines and deliverables, allowing you to concentrate on technical excellence
  • ML Ops and Advanced Data Engineering:
  • Establish ML Ops practices within the data engineering framework, focusing on automation, monitoring, and optimization of machine learning pipelines
  • Data Quality and Governance:
  • Implement and maintain data quality frameworks, ensuring the accuracy, consistency, and reliability of data across the platform
  • Drive data governance initiatives, including data cataloguing, lineage tracking, and adherence to security and compliance standards

Requirements

Experience:

  • 3+ years of experience in data engineering, with a proven track record in building and maintaining data platforms, preferably on AWS
  • Strong proficiency in Python, experience in SQL and PostgreSQL. PySpark, Scala or Java is a plus
  • Familiarity with Databricks and the Delta Lakehouse concept
  • Experience mentoring or leading junior engineers is highly desirable

Skills:

  • Deep understanding of cloud-based data architectures and best practices
  • Proficiency in designing, implementing, and optimizing ETL/ELT workflows
  • Strong database and data lake management skills
  • Familiarity with ML Ops practices and tools, with a desire to expand skills in this area
  • Excellent problem-solving abilities and a collaborative mindset

Nice to Have:

  • Familiarity with containerization and orchestration tools (e.g., Docker, Kubernetes)
  • Knowledge of machine learning pipelines and their integration with data platforms

Great training and career development opportunities exist for the right candidate.

Basic salary £60-65,000 + excellent benefits

Office based in Northumberland. Fully remote working available


  • Senior Data Engineer

    2 weeks ago


    Northumberland, Northumberland, United Kingdom Tech4 Full time

    Senior Data Engineer - Python / Data Pipelines / Data Platform / AWS - is required by fast growing, highly successful and and tech focused organisation. About the job You will play a crucial role in designing, building, and maintaining their data platform, with a strong emphasis on streaming data, cloud infrastructure, and machine learning operations. ...

  • Senior Data Engineer

    2 weeks ago


    Northumberland, United Kingdom Tech4 Full time

    Senior Data Engineer - Python / Data Pipelines / Data Platform / AWS - is required by fast growing, highly successful and and tech focused organisation.About the jobYou will play a crucial role in designing, building, and maintaining their data platform, with a strong emphasis on streaming data, cloud infrastructure, and machine learning operations.Key...

  • Senior Data Engineer

    2 weeks ago


    Northumberland, United Kingdom Tech4 Full time

    Senior Data Engineer - Python / Data Pipelines / Data Platform / AWS - is required by fast growing, highly successful and and tech focused organisation.About the jobYou will play a crucial role in designing, building, and maintaining their data platform, with a strong emphasis on streaming data, cloud infrastructure, and machine learning operations.Key...

  • Senior Data Engineer

    2 weeks ago


    Northumberland, United Kingdom Tech4 Full time

    Senior Data Engineer - Python / Data Pipelines / Data Platform / AWS - is required by fast growing, highly successful and and tech focused organisation. About the job You will play a crucial role in designing, building, and maintaining their data platform, with a strong emphasis on streaming data, cloud infrastructure, and machine learning operations. Key...

  • Senior Data Engineer

    2 weeks ago


    Northumberland, United Kingdom Tech4 Full time

    Senior Data Engineer - Python / Data Pipelines / Data Platform / AWS - is required by fast growing, highly successful and and tech focused organisation.About the jobYou will play a crucial role in designing, building, and maintaining their data platform, with a strong emphasis on streaming data, cloud infrastructure, and machine learning operations.Key...


  • Northumberland, United Kingdom Tech4 Full time

    Tech4 is seeking a Senior AWS Data Specialist to join our team.About the JobThis role involves designing, building, and maintaining our data platform with a focus on streaming data, cloud services, and machine learning operations.Key Responsibilities:Data Pipeline Optimization:Design and implement scalable and efficient data pipelines using AWS...


  • Northumberland, United Kingdom Tech4 Full time

    We are looking for a skilled Data Platform Engineer to join our team at Tech4.About the JobThis position plays a key role in designing, building, and maintaining our data platform with a focus on streaming data, cloud services, and machine learning operations.Key Responsibilities:Develop and Maintain Data Pipelines:Design and implement scalable and efficient...

  • Design Engineer

    3 days ago


    Northumberland, United Kingdom MTrec Recruitment Full time

    **Rewards and Benefits on Offer;** - Competitive salary. - Career progression opportunities. - Stable growing organisation. - Permanent employment. - Company pension & benefits. - Immediate start. **MTrec’s New Opportunity;** MTrec Technical are proudly representing our prestigious engineering client, based in Northumberland, with their plans for growth...


  • Northumberland, United Kingdom H9 Technical Recruitment Full time

    H9 Technical are recruiting for a Performance & Production Support Engineer in Northumberland area. **Overview** **Role & Responsibilities** Reporting to the Engineering Manager, the key responsibilities and duties will include: - Co-ordinating of generation production plans, providing a key interface with EPUKI Traders, Operational, Maintenance and...


  • Northumberland, Northumberland, United Kingdom Sellick Partnership Full time €45,000

    Permanent £Private medical, company performance bonus Sellick Partnership are partnering with a well-known multi-national to recruit for an Infrastructure Analyst on a permanent basis to start in the New Year. You will have the opportunity to work on project specific work as well as being the day to day level 3 escalation point for on-site engineers. ...


  • Northumberland, United Kingdom Theo James Recruitment Ltd Full time

    **Import & Export Coordinator (Northumberland) - £25,000 - £27,000** *Genuine career progression within the business *Fantastic, enhanced benefits across the board **The Company**: A well-established Manufacturing company are looking to expand their Logistics department with an experienced Import & Export Coordinator. **Benefits** - Genuine career...


  • Prestwick, Northumberland, United Kingdom Talent84 LTD Full time

    Senior Administrator, Ponteland Salary circa £30,000 per annum. We are seeking an experienced Senior Administrator to provide full executive support to the Managing Director of our dynamic client. This is a full-time, office-based role where your presence and collaboration with the MD and various teams will be essential. Key Responsibilities: *...

  • Senior Administrator

    2 weeks ago


    Prestwick, Northumberland, United Kingdom CV-Library Full time

    Senior Administrator, Ponteland Salary circa £30,000 per annum. We are seeking an experienced Senior Administrator to provide full executive support to the Managing Director of our dynamic client. This is a full-time, office-based role where your presence and collaboration with the MD and various teams will be essential. Key Responsibilities: *...


  • Boulmer, Northumberland, United Kingdom Mainstay Industrial Full time

    Mainstay Industrial is seeking a highly skilled Senior M&E Engineer to join our team. As a key member of our engineering department, you will play a crucial role in ensuring the smooth operation of our facilities.The successful candidate will have a strong background in maintenance and engineering, with experience in planning, coordinating, and implementing...

  • Quality Manager

    1 month ago


    Northumberland, United Kingdom Randstad Construction and Property Full time

    My client, an industry leading tier one civil engineering and construction firm are looking for a Quality Engineer to join their Rail division on a temporary contract in the North East of England. This is a fantastic opportunity to be part of one of the most prestigious rail projects in the North East of England. Experience within the civil engineering...

  • Shift Manager

    6 days ago


    Northumberland, United Kingdom H9 Technical Recruitment Full time

    H9 Technical are hiring for a **Shift Manager** to work at our clients Biomass Power Station in Northumberland. The plant has been recently converted from Coal to Biomass and is now capable of producing ~420MW The role will involve overseeing the day to day operational activities on site, ensuring the plant is achieving its commercial obligations. **Role &...


  • Boulmer, Northumberland, United Kingdom Mainstay Industrial Full time

    About the Role:This is an exciting opportunity to join Mainstay Industrial as a Senior M&E Engineer. The successful candidate will be responsible for maintaining and improving the company's mechanical and electrical systems. This will involve conducting regular inspections, performing repairs and maintenance, and developing strategies to reduce downtime and...

  • Quality Supervisor

    2 days ago


    Northumberland, United Kingdom MTrec Full time

    **Company Description**: Company Description Our client is a market leader in their sector. Due to growth, they are now looking for a motivated Quality Supervisor who is looking to grasp an outstanding career opportunity, and help a business grow and improve. **Job Specification**: The Role - In this position the Quality Supervisor will support the Quality...

  • Admin Assistant

    2 days ago


    Northumberland, United Kingdom Hays Specialist Recruitment Limited Full time

    School Administrator required for Northumberland Primary School Your new school Your new 'Good' primary school in Northumberland are currently looking for a school administrator/receptionist on a full time, temporary basis long term. What will Hays do for you - Give you access to a consultant remotely - Video Interview - Potential opportunity to work...

  • Administrator

    1 day ago


    Northumberland, United Kingdom Hays Specialist Recruitment Limited Full time

    Administrator required for Northumberland School Your new school Your new 'Good' primary school in Northumberland are currently looking for a school administrator to join them on a full time, permanent basis from September What will Hays do for you - Give you access to a consultant remotely - Video Interview - Potential opportunity to work remotely - Access...