Senior Data Engineer

2 weeks ago


Cramlington, United Kingdom Tech4 Ltd Full time

Senior Data Engineer - Python / Data Pipelines / Data Platform / AWS - is required by fast growing, highly successful and tech focused organisation.

About the job

You will play a crucial role in designing, building, and maintaining their data platform, with a strong emphasis on streaming data, cloud infrastructure, and machine learning operations.

Key Responsibilities:

* Architect and Implement Data Pipelines:

* Design, develop, and maintain scalable and efficient data pipelines

* Optimize ETL processes to ensure seamless data ingestion, processing, and integration across various systems

* Streaming Data Platform Development:

* Lead the development and maintenance of a real-time data streaming platform using tools like Apache Kafka, Databricks, Kinesis.

* Ensure the integration of streaming data with batch processing systems for comprehensive data management

* Cloud Infrastructure Management:

* Utilize AWS data engineering services (including S3, Redshift, Glue, Kinesis, Lambda, etc.) to build and manage our data infrastructure

* Continuously optimize the platform for performance, scalability, and cost-effectiveness

* Communications:

* Collaborate with cross-functional teams, including data scientists and BI developers, to understand data needs and deliver solutions

* Leverage the project management team to coordinate project, requirements, timelines and deliverables, allowing you to concentrate on technical excellence

* ML Ops and Advanced Data Engineering:

* Establish ML Ops practices within the data engineering framework, focusing on automation, monitoring, and optimization of machine learning pipelines

* Data Quality and Governance:

* Implement and maintain data quality frameworks, ensuring the accuracy, consistency, and reliability of data across the platform

* Drive data governance initiatives, including data cataloguing, lineage tracking, and adherence to security and compliance standards

Requirements

Experience:

* 3+ years of experience in data engineering, with a proven track record in building and maintaining data platforms, preferably on AWS

* Strong proficiency in Python, experience in SQL and PostgreSQL. PySpark, Scala or Java is a plus

* Familiarity with Databricks and the Delta Lakehouse concept

* Experience mentoring or leading junior engineers is highly desirable

Skills:

* Deep understanding of cloud-based data architectures and best practices

* Proficiency in designing, implementing, and optimizing ETL/ELT workflows

* Strong database and data lake management skills

* Familiarity with ML Ops practices and tools, with a desire to expand skills in this area

* Excellent problem-solving abilities and a collaborative mindset

Nice to Have:

* Familiarity with containerization and orchestration tools (e.g., Docker, Kubernetes)

* Knowledge of machine learning pipelines and their integration with data platforms

Great training and career development opportunities exist for the right candidate.

Basic salary £60-65,000 + excellent benefits

Office based in Northumberland. Fully remote working available


  • Senior Data Engineer

    2 weeks ago


    Cramlington, United Kingdom CV-Library Full time

    Senior Data Engineer - Python / Data Pipelines / Data Platform / AWS - is required by fast growing, highly successful and tech focused organisation. About the job You will play a crucial role in designing, building, and maintaining their data platform, with a strong emphasis on streaming data, cloud infrastructure, and machine learning operations. Key...

  • Senior Data Engineer

    4 months ago


    Cramlington, United Kingdom Printed.com Full time

    You will play a crucial role in designing, building, and maintaining our data platform, with a strong emphasis on streaming data, cloud infrastructure, and machine learning operations.Key Responsibilities: Architect and Implement Data Pipelines: Design, develop, and maintain scalable and efficient data pipelines. Optimize ETL processes to ensure seamless...


  • Cramlington, Northumberland, United Kingdom Tech4 Ltd Full time

    About the PositionTech4 Ltd is seeking a talented Senior Data Engineer to join our team. The successful candidate will be responsible for designing, building, and maintaining our cloud-based data architecture, focusing on real-time data streaming and machine learning operations.Responsibilities:Cloud Architecture:Design and build our data infrastructure...


  • Cramlington, Northumberland, United Kingdom Printed Full time

    About the JobWe're looking for a talented Cloud Native Data Solutions Expert to help us design, build, and maintain our cloud-based data platform at Printed. As an AWS Data Platform Engineer, you'll be responsible for architecting and implementing scalable and efficient data pipelines, designing real-time data streaming platforms, and collaborating with...

  • Cloud Data Architect

    4 weeks ago


    Cramlington, Northumberland, United Kingdom Printed Full time

    Job DescriptionWe are seeking a highly skilled Senior Data Engineer to join our team at Printed. As a key member of our data engineering team, you will play a critical role in designing, building, and maintaining our data platform.Key ResponsibilitiesData Pipeline Architecture and ImplementationDesign, develop, and maintain scalable and efficient data...


  • Cramlington, Northumberland, United Kingdom Printed Full time

    About the RoleWe are seeking an experienced AWS Data Platform Engineer to join our team at Printed. The successful candidate will play a crucial role in designing, building, and maintaining our cloud-based data platform with a strong emphasis on streaming data, cloud infrastructure, and machine learning operations.ResponsibilitiesArchitect and implement...


  • Cramlington, Northumberland, United Kingdom Deerfoot Recruitment Solutions Full time

    Project Leadership OpportunityWe are looking for a highly experienced Senior Engineering Manager to lead impactful technical projects.The ideal candidate will have a strong foundation in electronics or electrical engineering.


  • Cramlington, Northumberland, United Kingdom Printed Full time

    About the CompanyWe're Printed, a dynamic company seeking a skilled AWS Data Platform Engineer to join our team. As a key member of our data engineering team, you'll play a vital role in designing, building, and maintaining our cloud-based data platform, with a focus on streaming data, cloud infrastructure, and machine learning...


  • Cramlington, Northumberland, United Kingdom MTrec Technical Full time

    About the RoleMTrec Technical is seeking a highly skilled Senior Product Development Engineer to join their team. As a senior member, you will be responsible for designing and developing innovative products that meet customer needs.The successful candidate will have a strong background in mechanical engineering, excellent problem-solving skills, and...

  • Senior Production Lead

    13 hours ago


    Cramlington, Northumberland, United Kingdom CDM Recruitment Ltd - Engineering & Manufacturing Specialists Full time

    About the RoleCWM Recruitment Ltd - Engineering & Manufacturing Specialists are looking for a senior production lead to join our team in Cramlington. This exciting opportunity offers the chance to work in a fast-paced environment, leveraging your expertise to drive business growth and success.The successful candidate will have a strong background in...


  • Cramlington, United Kingdom Baker Hughes Full time

    Senior Physicist -Magnetics Would you like to shape our strategy for data? Are you passionate about leading a successful team?   Join our Team! Would you like to be part of a team delivering cutting edge physics concepts and algorithms for intelligent systems solutions to a global audience? Would you like to work with well-established...


  • Cramlington, Northumberland, United Kingdom Baltic Recruitment Services Ltd Full time

    We are currently looking for a Senior Electrical Engineer to join our team in Cramlington. As a key member of our maintenance team, you will be responsible for diagnosing and resolving electrical faults, ensuring minimal downtime and maintaining the highest standards of quality.Your primary focus will be on maintaining and repairing electrical systems,...


  • Cramlington, Northumberland, United Kingdom bioM?rieux BV Full time

    Job DescriptionAt bioM'rieux BV, we are looking for an experienced Mechanical Engineer to lead our maintenance strategy and drive best practice compliance in our facilities maintenance program.This is a key role in our engineering team, where you will be responsible for driving strategic improvements to the department's maintenance program. You...

  • Senior Specialist

    7 months ago


    Cramlington, United Kingdom Organon Full time

    **The position** The Senior Specialist, Supplier Resiliency will report directly to the Director Third Party Risk & Compliance and will be responsible for leading the Third Party Resilience program for Organon. Responsibilities will include re-defining the model for Product Risk mapping, N-Tier suppliers mapping, suppliers risk identification and reporting...


  • Cramlington, Northumberland, United Kingdom MTrec Commercial Careers Full time

    Job Title: Senior Financial AnalystWe are seeking an experienced Senior Financial Analyst to join our team on a permanent basis.About the Role:Prepare monthly management accounts, short-term forecasts, and budgets in line with group reporting formats and deadlines.Review and update standard product costs regularly.Set standard costs for new products.Analyse...


  • Cramlington, Northumberland, United Kingdom MTrec Technical Full time

    Company Overview">MTrec Technical is a leading manufacturer in their field, known for innovation and market leadership. They are seeking a highly skilled Design Engineer to support the design of new products.">About the Role">The ideal candidate will have experience in designing and developing products, providing technical support to the team and all areas...


  • Cramlington, United Kingdom Talent Solution Partners Full time

    Electrical Maintenance Engineer £41,000 + training + career progression pathways A leading FM provider, with a global presence, are currently looking for an Electrical Maintenance Engineer for one of their sites based in the Newcastle area.  This is a fantastic opportunity to join a talented engineering team on a well operated site which has a track...


  • Cramlington, United Kingdom CV-Library Full time

    Electrical Maintenance Engineer  £41,000 + training + career progression pathways  A leading FM provider, with a global presence, are currently looking for an Electrical Maintenance Engineer for one of their sites based in the Newcastle area.  This is a fantastic opportunity to join a talented engineering team on a well operated site...


  • Cramlington, Northumberland, United Kingdom CDM Recruitment Full time

    Senior Portfolio ManagerWe're looking for an experienced Senior Portfolio Manager to lead our product portfolio strategy at CDM Recruitment. As a key member of our commercial planning team, you'll be responsible for driving business growth through effective product lifecycle management.In this role, you'll work closely with sales and product development...


  • Cramlington, United Kingdom Organon Full time

    **The Position** **Locations**: U.S. - East Coast or Central time zones Europe - UK, Ireland, Netherlands or Lisbon, Portugal only. We show up every day to take the promise of Organon and turn it into a reality, with a call to create what has been missing in women's healthcare. We're here hoping to make a difference in improving everyday health for women...