Big Data Engineer

1 week ago

London, Greater London, United Kingdom Gaming Innovation Group Full time
Big Data Engineer

Department: Technology

Employment Type: Full Time

Location: United Kingdom


Description

As part of the Big Data Team, you will be using cutting edge technologies to work on collating data across different sources within GiG. Sources are all in real time, and you will be supporting an event driven micro service lambda architecture.

Your focus is to work with the current agile team and contribute to the platform's objectives and continue being an enabler to many other data technology stakeholders such as the Business Intelligence, Data Science and Quality Assurance teams. You will be up for a challenge, to continue maintaining and adding new features to a platform that ingests up to 5000 messages per second across multiple products such as Lottery, Sports and Casino. 

Reporting to the Lead Big Data Engineer and supported by the Big Data Architect to follow best practices, you'll be guided into delivering world class product increments that support our Tier 1 Platform. 

Key Responsibilities
  • Participate in all agile scrum meetings such as daily standups, refinement sessions, retro sessions, etc
  • Maintaining the data platform by means of daily reconciliations, data reloads, tackling support tickets, etc.
  • Being able to come up with new ideas to improve existing products and services.
  • Implementation of bug fixes and enhancements within the data platform.
  • Performing code reviews for other engineers.
  • Taking ownership of releases where necessary
  • Being able to communicate with the respective stakeholders, making sure that all information is transmitted correctly and to the right audience.
  • Take ownership of complex, large-scale initiatives, including data migration projects, ensuring successful planning, execution, and delivery.


Skills, Knowledge & Expertise You're really awesome at:
  • Object oriented programming (Java)
  • Data modelling using any database technologies
  • ETL processes (ETLs are oldschool, we transfer in memory now) and experience with Apache Spark or Apache NiFi
  • Applied understanding of CI\CD in change management
  • Dockerised applications
  • Used distributed version control systems
  • Excellent team player
  • Meticulous and passionate about your work

You're also good at:
  • Functional oriented programming (experience with Python, Scala)
  • Analytical mindset
  • Low latency databases such as ClickHouse
  • Hold a Bachelor's Degree in Computer Science or equivalent
  • Can collaborate in scoping exercises
  • Mentoring skills
  • Have worked with both Linux and Windows environments.
  • Unix/Linux


Job Benefits
  • Great career development opportunities
  • 100% remote or hybrid working model
  • International Health Insurance
  • Health and Wellbeing Package (350 EUR per year)
  • Birthday Day Off
  • Me Time - 1 day off per year
  • Lunches in the office for free

  • Senior Associate Data Engineering Big Data

    2 weeks ago

    London, Greater London, United Kingdom Publicis Sapient Full time £800,000 - £1,000,000 per year

    Job DescriptionSenior Associate Data Engineering L1As a Senior Associate Data Engineer, you will be responsible for designing, developing, and maintaining scalable Big Data solutions. You will work with large datasets, real-time processing frameworks, and cloud-based data platforms to enable data-driven decision-making for our clients.QualificationsSkills &...

  • Big Data Developer

    4 days ago

    London, Greater London, United Kingdom Information Tech Consultants Full time £100,000 - £150,000 per year

    Job Title:Junior Big Data Developer (Urgent hiring)Location:UKExperience:2-6 yearsEducation:Bachelors in Science (IT/Computer Science/Engineer)Employment Type:Full-TimeUK based candidates only.Role Overview:We are seeking an experiencedScala /Hadoop/ Big Data Developer. The role will focus on building and enhancing data-driven solutions, working within a...

  • Big Data Consultant

    2 weeks ago

    London, Greater London, United Kingdom Information Tech Consultants Full time £60,000 - £120,000 per year

    Job Title:Senior Big Data DeveloperLocation:UKExperience:5-8 yearsEducation:Bachelors in Science (IT/Computer Science/Engineer)Employment Type:Full-TimeUK based candidates only.Role Overview:We are seeking an experiencedScala /Hadoop/ Big Data Developer. The role will focus on building and enhancing data-driven solutions, working within a fast-paced...

  • Senior Big Data Engineer

    6 days ago

    London, Greater London, United Kingdom Global Relay Full time £80,000 - £120,000 per year

    Who we are:For over 20 years, Global Relay has set the standard in enterprise information archiving with industry-leading cloud archiving, surveillance, eDiscovery, and analytics solutions. We securely capture and preserve the communications data of the world's most highly regulated firms, giving them greater visibility and control over their information...

  • Senior Data Engineer

    2 days ago

    London, Greater London, United Kingdom Data Idols Full time £70,000 - £75,000 per year

    Senior Data EngineerSalary: £70,00 - £75,000Location: Hybrid - London - 1 day per weekWe are currently looking for a Senior Data Engineer to join our client's forward-thinking, collaborative, and technology-driven team. In this role, you'll take the lead in shaping and delivering modern cloud data solutions, working closely with a highly skilled Data & BI...

  • Big Data Solutions Architect

    6 days ago

    London, Greater London, United Kingdom Databricks Full time £60,000 - £100,000 per year

    REQ ID: CSQ326R394Location: LondonAs a Big Data Solutions Architect, in our Professional Services team you will work with clients on short to medium term customer engagements on their big data challenges using the Databricks platform. You will provide data engineering, data science, and cloud technology projects which require integrating with client...

  • Data Engineer Contractor

    3 days ago

    London, Greater London, United Kingdom Data Intellect Full time £60,000 - £100,000 per year

    Company Description We are looking for a highly motivated Data Engineer to join our dynamic team working with a tier 1 multinational investment bank. This role is ideal for someone who thrives in fast-paced environments, enjoys working on both BAU and project-based initiatives, and has a strong background in Python ideally within the FX domain. Please note,...

  • Big Data Solutions Architect

    5 days ago

    London, Greater London, United Kingdom Databricks Full time £100,000 - £120,000 per year

    Req ID: CSQ326R205We have 7 open positions based in our London office.As a Solutions Architect in our Professional Services team you will work with clients on short to medium-term customer engagements on their big data challenges using the Databricks Platform. You will provide data engineering, data science, and cloud technology projects which require...

  • Big Data Solutions Architect

    6 days ago

    London, Greater London, United Kingdom Databricks Full time £80,000 - £120,000 per year

    Req ID: CSQ326R205We have 7 open positions based in our London office.As a Solutions Architect in our Professional Services team you will work with clients on short to medium-term customer engagements on their big data challenges using the Databricks Platform. You will provide data engineering, data science, and cloud technology projects which require...

  • Analytics Engineer

    2 weeks ago

    London, Greater London, United Kingdom Data Idols Full time £45,000 - £55,000 per year

    Analytics EngineerContract - £450 - £550pd Location: London - 2 days per weekWe are currently looking for an Analytics Engineer to join our fast-paced, data-driven tech team within a global digital media environment. You'll play a crucial role in shaping how the business collects, models, and activates data across multiple commercial and editorial...