Data Engineer

2 weeks ago

London, United Kingdom Better Days Recruitment Full time

My client is a not-for-profit organisation focused on developing the blockchain/DLT ecosystem.


Due to ongoing growth they are now looking for an experienced Data Engineer to join their team.


This role is pivotal in managing, curating, optimising, and securing datasets specifically related to cryptocurrency discussions across various platforms. The ideal candidate will be adept in web scraping, data quality assurance using AI, data integration, ensuring data security and compliance, and maintaining detailed documentation.


Blockchain experience is not needed but a solid background with Python, AWS and web scraping is a must.


Role and Responsibilities

  • Data Collection: Identify relevant chat sources, groups, and forums on platforms discussing particular topics. Maintain and develop web scraping tools or APIs for periodic data extraction.
  • Data Quality Assurance: Develop and implement AI-based procedures for quality control of data and data sources to eliminate inaccuracies and anomalies. Create tools for monitoring data sources for changes and updates, adapting data collection and cleaning processes accordingly.
  • Data Integration: Collaborate with data scientists and analysts to integrate collected data into various projects and analysis tools. Ensure smooth data flow and integration with other data sources within the organisation.
  • Data Security and Compliance: Uphold the security and privacy of collected data in compliance with relevant regulations and company policies.
  • Documentation: Maintain clear and comprehensive documentation of data sources, collection methods, and workflows. Produce reports and documentation for both internal and external stakeholders as required.
  • Monitoring and Reporting: Develop and maintain systems to monitor the performance and health of data collection processes.


Key skills needed;

  • Bachelor’s degree in Computer Science, Data Science, or a related field.
  • Demonstrable experience in a data engineering role, with a focus on web scraping and data collection.
  • Proficient in programming languages such as Python, SQL.
  • Strong AWS experience
  • Knowledge of Data Structures and Databases
  • Knowledge of data privacy laws and compliance requirements.
  • Experience with Data Analytics/algorithms would be nice to have but training will be provided for the right candidate
  • Excellent communication and collaboration abilities.


Desirable

Experience with Typescript

Experience with big data technologies and cloud services.

Proficiency in AI and machine learning techniques for data quality assurance.


They are happy for this role to be predominately remote based with once or twice a month in the office

  • Data Engineer

    1 month ago

    London, United Kingdom Data Idols Full time

    **Data Engineer** Data Idols are excited to be working with one of the top 100 companies in the world to recruit a Data Engineer! A company which owns multiple brands and who are investing heavily into 'all things data' to revolutionise their extraordinary team. **Skills and Experience**: - Experience with Azure - Experience with SQL - Experience with...

  • Data Engineer

    1 month ago

    London, United Kingdom Data Idols Full time

    **Data Engineer** Data Idols are proud to be working with a growing veterinary business who are extremely data driven and have a great mission to use data to improve the welfare of animals. They have a revolutionary data team who want to be recognised as the UK's leadingveterinary group and are home to some of the best clinicians world-wide; would you like...

  • Data Engineer

    1 month ago

    London, United Kingdom Data Idols Full time

    **Data Engineer** Data Idols are proud to be working with a leading technology and engineering consultancy who have been running for decades globally. They work with a variety of clients across different industries on their biggest challenges. Would you like to work alongsidethis company, where you get the chance to work across a wide range of tools, tech,...

  • Data Engineer

    1 month ago

    City of London, United Kingdom Data Idols Full time

    **Senior Data Engineer** Data Idols are proud to be working with an iconic UK brand to assist them in hiring a Senior Data Engineer as they undertake a digital and data-driven journey that will transform the business. Data is at the heart of the organisation and forms a key partof their future plans. **The Opportunity** As a Senior Data Engineer you will be...

  • Senior Engineer, Data Engineering

    4 weeks ago

    London, United Kingdom Data Idols Full time

    Senior Data Engineer Location: London (Hybrid) Data Idols are working with one of the fast-growing companies in the logistics sector that are making the world a greener place. They are looking for a Senior Data Engineer to join their team within the Data Space. As a Senior Data Engineer, you will work alongside different data teams to design,...

  • Senior Engineer, Data Engineering

    4 weeks ago

    London, United Kingdom Data Idols Full time

    Senior Data Engineer Location: London (Hybrid) Data Idols are working with one of the fast-growing companies in the logistics sector that are making the world a greener place. They are looking for a Senior Data Engineer to join their team within the Data Space. As a Senior Data Engineer, you will work alongside different data teams to design,...

  • Senior Engineer, Data Engineering

    2 weeks ago

    London, United Kingdom Data Idols Full time

    Job Description Senior Data Engineer Location: London (Hybrid) Data Idols are working with one of the fast-growing companies in the logistics sector that are making the world a greener place. They are looking for a Senior Data Engineer to join their team within the Data Space. As a Senior Data Engineer, you will work alongside different data teams to...

  • Senior Data Engineer

    4 weeks ago

    London, United Kingdom Data Idols Full time

    **Senior Data Engineer** Data Idols are excited to be working with a well-funded startup who are moving into their scale-up phase. We are working to recruit the first Data Engineer, you will be involved in architecting and building their greenfield data platform. In this excitingand dynamic role, you will be involved from the very beginning, selecting the...

  • Lead Engineer/Data Engineer

    2 months ago

    London, United Kingdom Abs Data Full time

    Lead Data Engineer Do you enjoy creating valuable insights with data? Do you want to play a key role in transforming our firm into an agile organization? Being agile will make us more responsive, more adaptable, and ultimately more innovative. We’re looking for a Data Engineer to: ~ transform data into valuable insights that inform business...

  • Lead Engineer/Data Engineer

    2 days ago

    London, United Kingdom Abs Data Full time

    Lead Data Engineer Do you enjoy creating valuable insights with data? Do you want to play a key role in transforming our firm into an agile organization? Being agile will make us more responsive, more adaptable, and ultimately more innovative. We’re looking for a Data Engineer to: ~ transform data into valuable insights that inform business...

  • Senior Data Engineer

    1 month ago

    London, United Kingdom Data Idols Full time

    **Senior Data Engineer** Data Idols are proud to be working with a company who has a mission to improve the lives of learners globally, who want to build a more sustainable future everyone can reach their potential in. Do you want to be a part of a change into the sustainable future? **The Opportunity** Within this role as a Senior Data Engineer you will be...

  • Senior Engineer, Data Engineering

    2 months ago

    London, United Kingdom Data Idols Full time

    Senior Data Engineer Location: London, 3/2 days in the office per week Data Idols are currently working with a company that focus on the european marketplace for transport who are looking to make the future greener! Within this role as a Senior Data Engineer, you will be monitoring, designing and implementing a datawarehouse, whilst working with...

  • Senior Engineer, Data Engineering

    2 months ago

    London, United Kingdom Data Idols Full time

    Senior Data Engineer Location: London, 3/2 days in the office per week Data Idols are currently working with a company that focus on the european marketplace for transport who are looking to make the future greener! Within this role as a Senior Data Engineer, you will be monitoring, designing and implementing a datawarehouse, whilst working with...

  • Senior Engineer, Data Engineering

    2 days ago

    London, United Kingdom Abs Data Full time

    Data Engineer With more than 55,000 people in 40+ countries, working at Jacobs offers an exciting range of opportunities to develop your career within a supportive and diverse team who always strive to do the right thing for our people, clients and communities. People are Jacobs’ greatest asset, and we offer a competitive package to retain and attract...

  • Senior Data Engineer

    1 month ago

    London, United Kingdom Data Idols Full time

    **Senior Data Engineer** Data Idols are proud to be working with a leading technology and engineering consultancy who have been running for decades globally. They work with a variety of clients across different industries on their biggest challenges. Would you like to work alongsidethis company, where you get the chance to work across a wide range of tools,...

  • Data Engineer

    4 weeks ago

    City of London, United Kingdom Data Idols Full time

    **Data Engineer - Biotechnology** Data Idols are working with an exciting and innovative biotechnology startup who are working to assist biopharma teams to find better medicines and drugs quicker and more efficiently. **The Opportunity** This is an opportunity for a Data Engineer to join a small team and have a big impact. As a Data Engineer you will...

  • Lead Data Engineer

    1 month ago

    City of London, United Kingdom Data Idols Full time

    **Lead Data Engineer** Data Idols are proud to be working with an iconic UK brand to assist them in hiring a Lead Data Engineer as they undertake a digital and data-driven journey that will transform the business. Data is at the heart of the organisation and forms a key part oftheir future plans. **Skills and Experience**: - Awareness and understanding of...

  • Software Engineering Manager

    6 days ago

    London, United Kingdom Data Idols Full time

    **Software Engineering Manager** We're working with a well funded scale up with a strong presence in the UK and USA. Due to growth they are now looking to recruit a Software Engineering Manager, the role will lead a team of 4 Data Engineers and report to the CTO. The business is based ona technology product used by millions, as the Software Engineering...

  • AI Data Science Engineer

    4 days ago

    London, United Kingdom Abs Data Full time

    AI Squad – Data Engineer KPMG Azure Data EngineerBase Location: London – (Hybrid working – 2 days in the offices)The KPMG Audit Technology team is dedicated to building cutting-edge solutions in close collaboration with the Audit function. We blend audit expertise with the latest technology, enabling us to understand the challenges our customers...

  • AI Data Science Engineer

    1 day ago

    London, United Kingdom Abs Data Full time

    AI Squad – Data Engineer KPMG Azure Data EngineerBase Location: London – (Hybrid working – 2 days in the offices)The KPMG Audit Technology team is dedicated to building cutting-edge solutions in close collaboration with the Audit function. We blend audit expertise with the latest technology, enabling us to understand the challenges our customers...