Spark Architect

1 week ago

London, Greater London, United Kingdom Axiom Software Solutions Limited Full time
About the Role

Axiom Software Solutions Limited is seeking a highly skilled Data Integration Specialist to join their Global Data Technology (GDT) team. As a Data Integration Specialist, you will play a key role in driving the upgrade of Data Integration to PySpark and working on an Enterprise scale Cloud infrastructure and Cloud Services in one of the Clouds (GCP).

Key Responsibilities

  • Refactor Legacy ETL code from DataStage to PySpark using Prophecy low-code, no-code, and available converters.
  • Analyze Spark code failures through Spark Plans and make correcting recommendations.
  • Review PySpark and Spark SQL jobs and make performance improvement recommendations.
  • Understand DataFrames / Resilient Distributed Data Sets and make corrective recommendations for memory-related problems.
  • Monitor Spark jobs using wider tools such as Grafana to identify Cluster level failures.
  • Demonstrate deep knowledge of how Cloudera Spark is set up and how the run-time libraries are used by PySpark code.

Requirements

  • At least 12 years of IT experience with deep understanding of component understanding around Spark Data Integration (PySpark scripting, variable setting, etc.), Spark SQL, Spark Explain plans.
  • Data Integration SME - ability to analyze Spark code failures through Spark Plans and make correcting recommendations.
  • Ability to traverse and explain the architecture you have been a part of and why any particular tool/technology was used.
  • Data Integration SME - ability to review PySpark and Spark SQL jobs and make performance improvement recommendations.
  • Data Integration SME - ability to understand DataFrames / Resilient Distributed Data Sets and make corrective recommendations for memory-related problems.
  • Monitoring - Spark jobs using wider tools such as Grafana to identify Cluster level failures.
  • Cloudera (CDP) Spark and how the run-time libraries are used by PySpark code.
  • Prophecy - high-level understanding of low-code, no-code Prophecy set up and its use to generate PySpark code.
  • Ready to work at least three days from Axiom Software Solutions Limited office and accept changes as per customer/Wipro policies.

Desirable Skills

  • Collaboration with multiple customer stakeholders.
  • Knowledge of working with Cloud Databases.
  • Excellent communication and solution presentation skills.

  • Senior Spark Solutions Architect

    3 weeks ago

    London, Greater London, United Kingdom Axiom Software Solutions Limited Full time

    JOB ADVERTISEMENT FOR Axiom Software Solutions LimitedWe are excited to present a Long-Term Opportunity to engage with Cutting-Edge Technology in a dynamic environment. As a leading player in the financial services sector, our client operates globally with a vast IT infrastructure that supports a multitude of data-driven initiatives.Role OverviewAs a Senior...

  • Senior Spark Solutions Architect

    3 weeks ago

    London, Greater London, United Kingdom Axiom Software Solutions Limited Full time

    Job OverviewWe are excited to present a unique opportunity to join Axiom Software Solutions Limited as a Senior Spark Solutions Architect. This role offers the chance to engage with advanced technologies within a dynamic environment, contributing to one of our most significant accounts.About the CompanyAxiom Software Solutions Limited is recognized for its...

  • Spark Architect

    4 days ago

    London, Greater London, United Kingdom Axiom Software Solutions Limited Full time

    About the RoleWe are seeking a highly skilled Spark Architect to join our team at Axiom Software Solutions Limited. As a key member of our Global Data Technology (GDT) team, you will be responsible for driving data integration upgrades to PySpark and working on an enterprise-scale cloud infrastructure.Key ResponsibilitiesDesign and implement data integration...

  • Spark Architect

    5 days ago

    London, Greater London, United Kingdom Axiom Software Solutions Limited Full time

    About the RoleWe are seeking a highly skilled Spark Architect to join our team at Axiom Software Solutions Limited. As a key member of our Global Data Technology (GDT) team, you will be responsible for driving data integration upgrades to PySpark and working on an enterprise-scale cloud infrastructure.Key ResponsibilitiesDesign and implement data integration...

  • Data Architect

    6 days ago

    London, Greater London, United Kingdom Empiric Full time

    Job Title: Senior Data ArchitectCompany: EmpiricJob Type: PermanentLocation: Remote (95%) with occasional visits to UK customer sitesAbout the Role:We are seeking a highly skilled Senior Data Architect to join our team at Empiric. As a Senior Data Architect, you will play a key role in architecting a large-scale data platform using Palantir Foundry, Spark,...

  • Spark Architect

    13 hours ago

    London, Greater London, United Kingdom Axiom Software Solutions Limited Full time

    About the RoleWe are seeking a highly skilled Spark Architect to join our team at Axiom Software Solutions Limited. As a key member of our Global Data Technology (GDT) team, you will be responsible for driving data integration upgrades to PySpark and working on an enterprise-scale cloud infrastructure.Key ResponsibilitiesDesign and implement data integration...

  • Spark Architect

    10 hours ago

    London, Greater London, United Kingdom Axiom Software Solutions Limited Full time

    About the RoleWe are seeking a highly skilled Spark Architect to join our team at Axiom Software Solutions Limited. As a key member of our Global Data Technology (GDT) team, you will be responsible for driving data integration upgrades to PySpark and working on an enterprise-scale cloud infrastructure.Key ResponsibilitiesDesign and implement data integration...

  • Data Architect

    3 days ago

    London, Greater London, United Kingdom Empiric Full time £80,000 - £135,000

    Job OverviewWe are seeking a highly skilled Data Architect to join our team at Empiric, working closely with one of our high-profile European Financial Services customers. As a key member of our team, you will be responsible for architecting a large-scale data platform using Palantir Foundry, Spark, and Python.Key ResponsibilitiesDesign and implement...

  • Data Architect

    3 days ago

    London, Greater London, United Kingdom Empiric Full time £80,000 - £135,000

    Job OverviewWe are seeking a highly skilled Data Architect to join our team at Empiric, working closely with one of our high-profile European Financial Services customers. As a key member of our team, you will be responsible for architecting a large-scale data platform using Palantir Foundry, Spark, and Python.Key ResponsibilitiesDesign and implement...

  • Data Architect

    4 days ago

    London, Greater London, United Kingdom Empiric Full time

    Job Title: Enterprise Data ArchitectCompany: EmpiricLocation: Remote (95%) with occasional visits to UK customer sitesJob Type: PermanentAbout the Role:We are seeking an experienced Enterprise Data Architect to join our team at Empiric. As a key member of our team, you will be responsible for architecting a large-scale data platform using Palantir Foundry,...

  • Data Architect

    4 days ago

    London, Greater London, United Kingdom Empiric Full time

    Job Title: Enterprise Data ArchitectCompany: EmpiricLocation: Remote (95%) with occasional visits to UK customer sitesJob Type: PermanentAbout the Role:We are seeking an experienced Enterprise Data Architect to join our team at Empiric. As a key member of our team, you will be responsible for architecting a large-scale data platform using Palantir Foundry,...

  • Azure Solutions Architect

    6 days ago

    London, Greater London, United Kingdom Hunter Bond Full time £120,000

    **Job Title:** Azure Architect**Job Summary:** Hunter Bond is seeking a highly skilled Azure Architect to lead the design and implementation of end-to-end solutions for our datalake initiatives. As a key member of our team, you will be responsible for providing technical leadership and expertise in architecting and designing scalable and extensible...

  • Senior Data Architect

    3 days ago

    London, Greater London, United Kingdom Ventula Consulting Full time

    Job Overview**Senior Data Engineer - Higher Education**Ventula Consulting is seeking a highly skilled Senior Data Engineer to join our team on a 6-month contract. This is an excellent opportunity to work with a well-established organization in a key project.The Senior Data Engineer will be responsible for supporting the development of a new data lake and...

  • Senior Data Architect

    3 days ago

    London, Greater London, United Kingdom Ventula Consulting Full time

    Job Overview**Senior Data Engineer - Higher Education**Ventula Consulting is seeking a highly skilled Senior Data Engineer to join our team on a 6-month contract. This is an excellent opportunity to work with a well-established organization in a key project.The Senior Data Engineer will be responsible for supporting the development of a new data lake and...

  • Cloud Data Architect

    6 days ago

    London, Greater London, United Kingdom Hunter Bond Full time £120,000

    **Job Title:** Azure Architect**Job Summary:** Hunter Bond is seeking a highly skilled Azure Architect to lead the design and implementation of end-to-end solutions for our datalake initiatives. As a key member of our engineering team, you will be responsible for providing technical leadership and driving improvements in design, processes, and implementation...

  • Cloud Solutions Architect

    6 days ago

    London, Greater London, United Kingdom Hunter Bond Full time

    About the RoleWe are seeking a highly skilled Cloud Solutions Architect to join our team at Hunter Bond. As a key member of our engineering team, you will be responsible for providing technical leadership in architecting and designing end-to-end solutions for our data lake initiatives.Key ResponsibilitiesProvide technical leadership in architecting and...

  • Data Architect

    7 hours ago

    London, Greater London, United Kingdom Altrata Full time

    Job SummaryAltrata is a global leader in people intelligence, providing data-driven insights to organizations. We are seeking a highly skilled Data Engineer to join our team.About the RoleAs a Data Engineer at Altrata, you will be responsible for designing, building, and maintaining large-scale data systems using Databricks, Python, Spark, R, and SQL. You...

  • Data Architect

    4 hours ago

    London, Greater London, United Kingdom Altrata Full time

    Job SummaryAltrata is a global leader in people intelligence, providing data-driven insights to organizations. We are seeking a highly skilled Data Engineer to join our team.About the RoleAs a Data Engineer at Altrata, you will be responsible for designing, building, and maintaining large-scale data systems using Databricks, Python, Spark, R, and SQL. You...

  • Big Data Architect

    2 days ago

    London, Greater London, United Kingdom Hunter Bond Full time

    Job Description**About Hunter Bond**Hunter Bond is seeking a highly skilled and experienced Big Data Architect to join our team in London. As a key member of our organization, you will play a crucial role in architecting and designing end-to-end solutions for our datalake initiatives.**Key Responsibilities**Provide technical leadership in architecting and...

  • Big Data Architect

    2 days ago

    London, Greater London, United Kingdom Hunter Bond Full time

    Job Description**About Hunter Bond**Hunter Bond is seeking a highly skilled and experienced Big Data Architect to join our team in London. As a key member of our organization, you will play a crucial role in architecting and designing end-to-end solutions for our datalake initiatives.**Key Responsibilities**Provide technical leadership in architecting and...