Python Software Engineer Data Lake

4 days ago

London, United Kingdom eFinancial Careers Full time

Python Software Engineer (Data Lake) London, Ropemaker Place The aim of this project is to: create a data lake - an environment which contains all the data that the company acquires, expose the ability to easily connect various data sets in it and finallydo complex data analysis. By creating easy access to the huge data lake, it will allow us to commercialize the data. This is a greenfield build of a data platform with cutting edge tech. The Platform team is responsible for an SDK and a backend REST service.Most of our code is written in Python, and we are branching out to some Java tools such as Presto and Spark. As a Software Engineer, You Will - Develop key backend components in a system for managing big data in the company. - Work as part of a remote teamusing Agile practices and methods (Scrum). We have team members in the UK, US, Poland and India. - Challenge any requirements in our planning meetings that do not make sense from a business or a technical point-of-view. - Deliver complex solutions to the ambiguousrequirements of our many users. - Support internal and external customers to understand their problems/requirements and adjust our design. - Be able to discuss your ideas for solving problems with other developers and business users. - Be proactive about identifyingany problems you see. As a Software Engineer, you will be expected to have: - Curiosity about why something is working even though it should not - Strong unit testing and troubleshooting skills - Eagerness to learn new things and passion for software development.If a customer comes to us wanting to know how to use a new Big Data tool with our datalake, the team will get hands-on experience with the tool. - High analytical and problem-solving skills - Full project life-cycle experience - Working level knowledge ofDocker - Good knowledge about common design patterns You will also need established experience in and/or knowledge of: - Python. We would also be interested in people with strong experience in Java/C#/Scala who are willing to learn Python. - Git - Jira - AWSservices e.g. S3 - Kubernetes - Pandas, Jupyter notebook, Spark, Dask.Inclusion and diversity are critical to the success of IHS Markit, and we actively encourage applications from people of all backgrounds. We are committed to providing equal employment opportunitywithout regard to race, color, religion, sex, sexual orientation, gender identity, age, national origin, disability, status as a protected veteran, or any other protected category. For more information on the many ways in which we enthusiastically supportinclusion and diversity efforts for both candidates and employees, please access our Inclusion & Diversity Statement here. We are proud to provide reasonable accommodations to applicants with disabilities. If you are interested in applying for employmentwith IHS Markit and need special assistance or an accommodation to use our website or to apply for a position, please contact or call +. Determination on requests for reasonable accommodation are considered on a case-by-case basis. This contact information(email and phone) is intended for application assistance and accommodation requests only. We are unable to accept resumes or provide information about application status through the phone number or email address above. Resumes are only accepted through theonline application process, and only qualified candidates will receive consideration and follow-up. IHS Markit maintains a substance-free workplace; employees may be asked to submit to a drug test (where permitted by law). In addition, in order to comply withapplicable federal, state, and local vaccine mandates (including those in place for US federal contractors), US employees may be required to provide proof of being fully vaccinated, unless they have been approved for an accommodation due to a medical reason,a sincerely held religious belief or another legally protected reason. T he company also participates in the E-Verify Program to confirm eligibility to work in the US. For information please click on the following links: IHS Markit Business Code of ConductRight to Work EEO is the Law EEO is the Law Supplement Pay Transparency Current Colleagues If you are currently employed by IHS Markit, please apply internally via the Workday internal careers site.

  • Python Software Engineer Data Lake

    4 days ago

    London, United Kingdom eFinancial Careers Full time

    **Python Software Engineer (Data Lake)** **London, Ropemaker Place** The aim of this project is to: create a data lake - an environment which contains all the data that the company acquires, expose the ability to easily connect various data sets in it and finally do complex data analysis. By creating easy access to the hugedata lake, it will allow us to...

  • Python Software Engineer

    22 hours ago

    London, United Kingdom Harnham Full time

    **Python Software Engineer - InsurTech / Fintech** **Up to £75,000 + bonus + pension** **London - Hybrid** An International InsurTech / Fintech organization is hiring a Python engineer to join their expanding London-based team. They are seeking a motivated and commercially experienced Python Engineer who is looking for a challenge in a dynamic and...

  • Senior Python Developer

    2 weeks ago

    London, Greater London, United Kingdom Harnham Full time

    About the Role:Harnham, a leading UK-based software company, seeks a skilled Cloud Data Engineer to join their product engineering organisation. As a key member of the team, you will be responsible for designing and implementing efficient data pipelines using Python, collaborating closely with the data science team to create a scalable data lake using...

  • Senior Data Lake Specialist

    2 weeks ago

    City of London, United Kingdom Wilson Brown Limited Full time

    Job OverviewWe are seeking an experienced Data Lake Specialist to join our team at Wilson Brown Limited in Central London. This is a senior role with a focus on testing data ingestion and transformation processes.The ideal candidate will have extensive experience working with C# or Python, as well as a strong understanding of AWS services such as s3, Glue...

  • Data Lakes Specialist

    6 days ago

    London, Greater London, United Kingdom N Consulting Limited Full time

    **Job Title:** Data Lakes Specialist**Location:** London, UKWe are seeking an experienced Data Lakes Specialist to join our team at N Consulting Limited. As a key member of our Big Data team, you will be responsible for designing, building, and managing scalable data lakes using the Hadoop ecosystem.**Key Responsibilities:Design and implement data ingestion...

  • Data Lake Engineer

    2 weeks ago

    London, Greater London, United Kingdom Conexus Full time

    Azure Data Architect Job OpportunityWe are looking for a skilled Azure Data Architect to join our team in London. This is a 6-month contract, with the possibility of further extension. As a member of our team, you will be responsible for designing and developing cloud-based data architectures using Microsoft Azure. Key responsibilities include developing and...

  • Principal Data Engineer Python Java Trading

    2 days ago

    London, United Kingdom eFinancialCareers Full time

    Principal Data Engineer (Architect Data Architecture Engineering Principal Lead Manager Java Python GCP AWS Cloud Trading Front Office Finance Financial Services Spark Kafka Agile Remote Data Lake Datalake Investment Banking Asset Management Manager) requiredby our fixed income trading software client in London. You MUST have the following: - Good...

  • Senior Python Software Engineer

    1 month ago

    London, Greater London, United Kingdom White Swan Data Full time

    White Swan Data OverviewWhite Swan Data is a rapidly growing team of mathematicians, data scientists, and software engineers striving to refine world-class betting models. Our work bridges three domains: betting and gambling, quantitative research, and software development.About the RoleWe are seeking a talented mid-level Python developer to join our team....

  • Senior Engineer, Data Engineering

    2 months ago

    London, United Kingdom Harnham Full time

    DATA ENGINEER LONDON BASED £This UK-based software firm is searching for a new Data Engineer, you will be joining their product engineering organisation, and will be responsible for building a new data lake in GCP. You will also be responsible for building new data pipelines in Python. They do this by collecting their client's Data and analysing this,...

  • Senior Engineer, Data Engineering

    4 months ago

    London, United Kingdom Harnham Full time €35,000 - €40,000

    DATA ENGINEER LONDON BASED £This UK-based software firm is searching for a new Data Engineer, you will be joining their product engineering organisation, and will be responsible for building a new data lake in GCP. You will also be responsible for building new data pipelines in Python. They do this by collecting their client's Data and analysing this,...

  • Senior Engineer, Data Engineering

    4 months ago

    London, United Kingdom Harnham Full time €35,000 - €40,000

    DATA ENGINEER LONDON BASED £This UK-based software firm is searching for a new Data Engineer, you will be joining their product engineering organisation, and will be responsible for building a new data lake in GCP. You will also be responsible for building new data pipelines in Python. They do this by collecting their client's Data and analysing this,...

  • Data Lake Architect

    3 weeks ago

    London, Greater London, United Kingdom Tesco Partners Full time

    Job OverviewTesco Partners is seeking a highly skilled Staff Analytics Engineer to join their Cyber Analytics team. This role requires a strong background in data engineering, analytics, and cybersecurity.About the RoleThis is an exciting opportunity for a talented individual to design, develop, and deploy scalable data lake and data warehouse solutions. The...

  • Data Lake Quality Assurance Specialist

    2 weeks ago

    City of London, United Kingdom Wilson Brown Limited Full time

    We are Wilson Brown Limited, an established Insurance firm based in Central London. We are currently seeking a highly skilled Data Lake Quality Assurance Specialist to join our team. As a key member of our quality assurance team, you will be responsible for testing data ingestion and transformation processes on our newly implemented Data Lake.The ideal...

  • Senior Data Software Engineer Java Trading Finance

    2 days ago

    London, United Kingdom eFinancialCareers Full time

    Senior Data Software Engineer (Architect Data Architecture Engineering Principal Lead Java Python GCP AWS Cloud Trading Front Office Finance Financial Services Spark Kafka Agile Datalake Data Lake Investment Banking Asset Manager Management) required byour fixed income trading software client in London. You MUST have the following: - Good experience as a...

  • Software Engineer Full Stack React Python

    3 weeks ago

    London, United Kingdom Qh4 Consulting Full time

    Software Engineer London – Investment Management – Front Office – React – Python - Node.js – Asset Management Are you a Software Engineer passionate about solving complex problems and delivering innovative solutions? We’re looking for a Software Engineer to join a dynamic team focused on building impactful tools and applications for a...

  • Software Engineer Full Stack React Python

    3 weeks ago

    London, United Kingdom Qh4 Consulting Full time

    Software Engineer London – Investment Management – Front Office – React – Python - Node.js – Asset Management Are you a Software Engineer passionate about solving complex problems and delivering innovative solutions? We’re looking for a Software Engineer to join a dynamic team focused on building impactful tools and applications for a...

  • Lead Data Engineer

    3 months ago

    London, United Kingdom Cornwallis Elt Full time

    Lead Data Engineer – Python, SQL, dbt, AWS, Dremio, Data Mesh, ETL, Data Pipelines, Data Warehousing, DWH, S3, Glue, Data Lake, Automation, London A Lead Data Engineer is sought after by a leading workplace pensions provider, to run their expanding data engineering team in their London City office. The team have recently built out a next-gen data lake...

  • Lead Data Engineer

    3 months ago

    London, UK, United Kingdom Cornwallis Elt Full time

    Lead Data Engineer – Python, SQL, dbt, AWS, Dremio, Data Mesh, ETL, Data Pipelines, Data Warehousing, DWH, S3, Glue, Data Lake, Automation, London A Lead Data Engineer is sought after by a leading workplace pensions provider, to run their expanding data engineering team in their London City office. The team have recently built out a next-gen data lake...

  • Data Engineer Position

    1 week ago

    London, Greater London, United Kingdom Harnham Full time £35,000 - £40,000

    Harnham is a UK-based software firm searching for a skilled Data Engineer to join their product engineering organisation. The successful candidate will be responsible for building and maintaining a new data lake in Google Cloud Platform (GCP) using Python.The ideal candidate will have extensive experience working with big data tools such as Apache Spark, GCP...

  • Lead Engineer/Data Engineer

    3 months ago

    London, United Kingdom Cornwallis Elt Full time

    Lead Data Engineer – Python, SQL, dbt, AWS, Dremio, Data Mesh, ETL, Data Pipelines, Data Warehousing, DWH, S3, Glue, Data Lake, Automation, London A Lead Data Engineer is sought after by a leading workplace pensions provider, to run their expanding data engineering team in their London City office. The team have recently built out a next-gen data lake...