Spark Architect

3 weeks ago

Leeds UK, West Yorkshire, United Kingdom PRACYVA Full time

Spark Architect / SME

Contract Role: Long term contract role but 6 months to begin with & its extendable

Location: Leeds, UK (min 3 days onsite)


JOB ADVERT FOR IJP


Exciting Long-Term Opportunity to Work on Cutting-Edge Technology in One of Client's Top Fastest-growing Accounts which has over 1900 associates working across India, UK, China, Hong Kong and Mexico. End client is one of the biggest financial services organizations in the world, with operations in more than 38 countries. It has an IT infrastructure of 200,000+ servers, 20,000+ database instances, and over 150 PB of data. As a Spark Architect, you will have the responsibility to refactor Legacy ETL code for example DataStage into PySpark using Prophecy low-code no-code and available converters. Converted code is causing failures/performance issues.


The End Client Account is looking for an enthusiastic Spark Architect with deep component understanding around Spark Data Integration (PySpark, scripting, variable setting etc.), Spark SQL, Spark Explain plans. Also able to analyse Spark code failures through Spark Plans and make correcting recommendations; able to review PySpark and Spark SQL jobs and make performance improvement recommendations; able to understand Data Frames / Resilient Distributed Data Sets and understand any memory related problems and make corrective recommendations; and able to monitor Spark jobs using wider tools such as Grafana to see whether there are Cluster level failures. As a Spark architect, who can demonstrate deep knowledge of how Cloudera Spark is set up and how the run time libraries are used by PySpark code.


Your benefits

As the Spark architect, you will have the opportunity to work with one of the biggest IT landscapes in the world. You can also look forward to opportunity to collaborate with senior leadership stakeholders in the End clients place, guiding technical team and drive overall program objectives.


Your responsibilities

As a Spark Architect you will be working for the largest British universal bank and financial services – GDT (Global Data Technology) Team, you will be responsible for:

  • Working on an Enterprise scale Cloud infrastructure and Cloud Services in one of the Clouds (GCP).

Drive Data Integration upgrade to PySpark.

Mandatory Skills

You need to have the below skills.

  • At least 12+ Years of IT Experience with Deep understanding of component understanding around Spark Data Integration (PySpark, scripting, variable setting etc.), Spark SQL, Spark Explain plans.Spark SME – Be able to analyse Spark code failures through Spark Plans and make correcting recommendations.
  • To be able to traverse and explain the architecture you have been a part of and why any particular tool/technology was used.Spark SME – Be able to review PySpark and Spark SQL jobs and make performance improvement recommendations.
  • Spark – SME Be able to understand Data Frames / Resilient Distributed Data Sets and understand any memory related problems and make corrective recommendations.

Monitoring –Spark jobs using wider tools such as Grafana to see whether there are Cluster level failures.Cloudera (CDP) Spark and how the run time libraries are used by PySpark code.Prophecy – High level understanding of Low-Code No-Code prophecy set up and its use to generate PySpark code.Ready to work at least three days from the clients Leeds (UK) office and accept changes as per customer/Wipro policies.

Good to have skills.

Ideally, you should be familiar with


Collaboration with multiple customer stakeholdersKnowledge of working with Cloud DatabasesExcellent communication and solution presentation skills.

  • Spark Architect

    4 days ago

    Leeds, United Kingdom PRACYVA Full time

    Spark Architect / SMECheck below to see if you have what is needed for this opportunity, and if so, make an application asap.Contract Role: Long term contract role but 6 months to begin with & its extendableLocation: Leeds, UK (min 3 days onsite)JOB ADVERT FOR IJPExciting Long-Term Opportunity to Work on Cutting-Edge Technology in One of Client's Top...

  • Spark Architect

    2 days ago

    Leeds, United Kingdom PRACYVA Full time

    Spark Architect / SMECheck below to see if you have what is needed for this opportunity, and if so, make an application asap.Contract Role: Long term contract role but 6 months to begin with & its extendableLocation: Leeds, UK (min 3 days onsite)JOB ADVERT FOR IJPExciting Long-Term Opportunity to Work on Cutting-Edge Technology in One of Client's Top...

  • Spark Architect

    1 week ago

    Leeds, United Kingdom PRACYVA Full time

    Spark Architect / SME Contract Role: Long term contract role but 6 months to begin with & its extendable Location: Leeds, UK (min 3 days onsite) JOB ADVERT FOR IJP Exciting Long-Term Opportunity to Work on Cutting-Edge Technology in One of Client's Top Fastest-growing Accounts which has over 1900 associates working across India, UK, China, Hong Kong...

  • Spark Architect

    3 weeks ago

    Leeds, United Kingdom PRACYVA Full time

    Spark Architect / SME Contract Role: Long term contract role but 6 months to begin with & its extendable Location: Leeds, UK (min 3 days onsite) JOB ADVERT FOR IJP Exciting Long-Term Opportunity to Work on Cutting-Edge Technology in One of Client's Top Fastest-growing Accounts which has over 1900 associates working across India, UK, China, Hong Kong...

  • Spark Architect

    1 month ago

    Leeds, United Kingdom PRACYVA Full time

    Spark Architect / SMEContract Role: Long term contract role but 6 months to begin with & its extendableLocation: Leeds, UK (min 3 days onsite)JOB ADVERT FOR IJPExciting Long-Term Opportunity to Work on Cutting-Edge Technology in One of Client's Top Fastest-growing Accounts which has over 1900 associates working across India, UK, China, Hong Kong and Mexico....

  • Spark Architect

    3 weeks ago

    Leeds, United Kingdom PRACYVA Full time

    Spark Architect / SMEContract Role: Long term contract role but 6 months to begin with & its extendableLocation: Leeds, UK (min 3 days onsite)JOB ADVERT FOR IJPExciting Long-Term Opportunity to Work on Cutting-Edge Technology in One of Client's Top Fastest-growing Accounts which has over 1900 associates working across India, UK, China, Hong Kong and Mexico....

  • Spark Architect

    3 weeks ago

    Leeds, United Kingdom PRACYVA Full time

    Spark Architect / SMEContract Role: Long term contract role but 6 months to begin with & its extendableLocation: Leeds, UK (min 3 days onsite)JOB ADVERT FOR IJPExciting Long-Term Opportunity to Work on Cutting-Edge Technology in One of Client's Top Fastest-growing Accounts which has over 1900 associates working across India, UK, China, Hong Kong and Mexico....

  • Spark Architect

    7 days ago

    Leeds, United Kingdom PRACYVA Full time

    Spark Architect / SME Contract Role: Long term contract role but 6 months to begin with & its extendable Location: Leeds, UK (min 3 days onsite) JOB ADVERT FOR IJP Exciting Long-Term Opportunity to Work on Cutting-Edge Technology in One of Client's Top Fastest-growing Accounts which has over 1900 associates working across India, UK, China, Hong Kong and...

  • Spark Architect

    2 months ago

    Leeds, United Kingdom Damia Group Ltd Full time

    Spark Architect - Leeds 3-days Onsite - 12-Month Contract - circa - £(Apply online only) for Inside IR35 Exciting opportunity for an experience Spark Architect to join a large Global Consultancy, you will be working to refactor Legacy ETL code using low-code no-code and available converters. The successful candidate will have extensive experience with...

  • Spark Architect

    23 hours ago

    Leeds, United Kingdom PRACYVA Full time

    Spark Architect / SMEContract Role: Long term contract role but 6 months to begin with & its extendableLocation: Leeds, UK (min 3 days onsite)The information below covers the role requirements, expected candidate experience, and accompanying qualifications.JOB ADVERT FOR IJPExciting Long-Term Opportunity to Work on Cutting-Edge Technology in One of Client's...

  • Spark Solutions Architect

    3 weeks ago

    Leeds, Leeds, United Kingdom Pracyva LTD Full time £650

    Overview: Pracyva LTD is seeking a skilled Spark Solutions Architect to lead the transformation of legacy ETL processes into modern PySpark implementations utilizing low-code and no-code methodologies. The ideal candidate will possess a deep understanding of Spark architecture and its components.Key Responsibilities:Analyze and troubleshoot converted code...

  • Cloud Data Architect

    3 days ago

    Leeds, Leeds, United Kingdom PRACYVA Full time

    Job DescriptionJob Title: Spark ArchitectJob Type: ContractLocation: Remote (with occasional on-site work)Job Description:We are seeking an experienced Spark Architect to join our team at PRACYVA. As a Spark Architect, you will be responsible for designing and implementing scalable and efficient data integration solutions using PySpark.Key...

  • Spark Architect

    3 days ago

    Leeds, Leeds, United Kingdom PRACYVA Full time

    Job DescriptionJob Title: Spark ArchitectJob Type: Long-term contract roleLocation: Remote with occasional on-site workAbout the Role:We are seeking an experienced Spark Architect to join our team at PRACYVA. As a Spark Architect, you will be responsible for designing and implementing scalable data integration solutions using PySpark and Cloud...

  • Performance Architect

    2 days ago

    Leeds, Leeds, United Kingdom PRACYVA Full time

    Job Summary:PRACYVA is seeking a highly skilled Performance Architect - Spark Expert to join our team. As a key member of our Data Engineering team, you will be responsible for driving data integration upgrades to PySpark, ensuring the performance and efficiency of our data processing systems.Key Responsibilities:Drive data integration upgrades to PySpark,...

  • Spark Architect

    2 days ago

    Leeds, Leeds, United Kingdom PRACYVA Full time

    About the RoleWe are seeking a highly skilled Spark Architect to join our team at PRACYVA. As a Spark Architect, you will be responsible for designing and implementing scalable and efficient data integration solutions using PySpark.Key ResponsibilitiesDesign and implement data integration pipelines using PySparkDevelop and maintain large-scale data...

  • Spark Architect

    16 hours ago

    Leeds, Leeds, United Kingdom PRACYVA Full time

    About the RoleWe are seeking a highly skilled Spark Architect to join our team at PRACYVA. As a key member of our Global Data Technology (GDT) Team, you will be responsible for driving data integration upgrades to PySpark and working on an enterprise-scale cloud infrastructure.Key ResponsibilitiesDesign and implement data integration solutions using PySpark...

  • Spark Architect

    13 hours ago

    Leeds, Leeds, United Kingdom PRACYVA Full time

    About the RoleWe are seeking a highly skilled Spark Architect to join our team at PRACYVA. As a key member of our Global Data Technology (GDT) Team, you will be responsible for driving data integration upgrades to PySpark and working on an enterprise-scale cloud infrastructure.Key ResponsibilitiesDesign and implement data integration solutions using PySpark...

  • Spark Architect

    2 days ago

    Leeds, Leeds, United Kingdom PRACYVA Full time

    About the RoleWe are seeking a highly skilled Spark Architect to join our team at PRACYVA. As a Spark Architect, you will be responsible for designing and implementing scalable and efficient data integration solutions using PySpark.Key ResponsibilitiesDesign and implement data integration pipelines using PySparkDevelop and maintain large-scale data...

  • Spark Architect

    1 month ago

    Leeds, United Kingdom Pracyva Ltd Full time

    Context: Legacy ETL code for example DataStage is being refactored into PySpark using Prophecy low-code no-code and available converters.Converted code is causing failures/performance issues.Skills: Spark Architecture component understanding around Spark Data Integration (PySpark, scripting, variable setting etc.), Spark SQL, Spark Explain plans. Spark SME ...

  • Spark Architect

    3 days ago

    Leeds, Leeds, United Kingdom PRACYVA Full time

    Job SummaryPRACYVA is seeking a highly skilled Spark Architect to join our team and lead the development of our Enterprise Data Integration platform using PySpark. As a key member of our Cloud Infrastructure team, you will be responsible for designing, implementing, and maintaining scalable and efficient data integration solutions on Cloud...