Spark Architect

Found in: Talent UK 2A C2 - 2 weeks ago

Sheffield, United Kingdom PRACYVA Full time

Spark Architect / SME

Contract Role- 6 months to begin with & its extendable

Location: Sheffield, UK (min 3 days onsite)

  • Context: Legacy ETL code for example DataStage is being refactored into PySpark using Prophecy low-code no-code and available converters.
  • Converted code is causing failures/performance issues.

Skills:

  • Spark Architecture – component understanding around Spark Data Integration (PySpark, scripting, variable setting etc.), Spark SQL, Spark Explain plans.
  • Spark SME – Be able to analyse Spark code failures through Spark Plans and make correcting recommendations.
  • Spark SME – Be able to review PySpark and Spark SQL jobs and make performance improvement recommendations.
  • Spark – SME Be able to understand Data Frames / Resilient Distributed Data Sets and understand any memory related problems and make corrective recommendations.
  • Monitoring – Be able to monitor Spark jobs using wider tools such as Grafana to see whether there are Cluster level failures.
  • Cloudera (CDP) – Knowledge of understanding how Cloudera Spark is set up and how the run time libraries are used by PySpark code.
  • Prophecy – High level understanding of Low-Code No-Code prophecy set up and its use to generate PySpark code.

  • Spark Architect

    Found in: Appcast UK C C2 - 1 week ago

    Sheffield, United Kingdom PRACYVA Full time

    Spark Architect / SMEContract Role- 6 months to begin with & its extendableLocation: Sheffield, UK (min 3 days onsite) Context: Legacy ETL code for example DataStage is being refactored into PySpark using Prophecy low-code no-code and available converters. Converted code is causing failures/performance issues.Skills: Spark Architecture – component...

  • Spark Architect

    Found in: Appcast Linkedin GBL C2 - 2 weeks ago

    Sheffield, United Kingdom PRACYVA Full time

    Spark Architect / SMEContract Role- 6 months to begin with & its extendableLocation: Sheffield, UK (min 3 days onsite) Context: Legacy ETL code for example DataStage is being refactored into PySpark using Prophecy low-code no-code and available converters. Converted code is causing failures/performance issues.Skills: Spark Architecture – component...

  • Spark Architect

    Found in: Jooble UK C2 - 2 weeks ago

    Sheffield, United Kingdom PRACYVA Full time

    Spark Architect / SME Contract Role- 6 months to begin with & its extendable Location: Sheffield, UK (min 3 days onsite) Context: Legacy ETL code for example DataStage is being refactored into PySpark using Prophecy low-code no-code and available converters. Converted code is causing failures/performance issues. Skills: Spark Architecture – component...

  • Spark Architect

    Found in: Whatjobs ES C2 - 4 days ago

    Sheffield, United Kingdom PRACYVA Full time

    Spark Architect / SME Contract Role- 6 months to begin with & its extendable Location: Sheffield, UK (min 3 days onsite) Context: Legacy ETL code for example DataStage is being refactored into PySpark using Prophecy low-code no-code and available converters. Converted code is causing failures/performance issues. Skills: Spark Architecture –...

  • Spark Architect

    2 weeks ago

    Sheffield, United Kingdom PRACYVA Full time

    Spark Architect / SME Contract Role- 6 months to begin with & its extendable Location: Sheffield, UK (min 3 days onsite) Context: Legacy ETL code for example DataStage is being refactored into PySpark using Prophecy low-code no-code and available converters. Converted code is causing failures/performance issues. Skills: Spark Architecture – component...

  • Spark Architect

    Found in: Appcast UK C2 - 1 week ago

    Sheffield, United Kingdom PRACYVA Full time

    Spark Architect / SMEContract Role- 6 months to begin with & its extendableLocation: Sheffield, UK (min 3 days onsite) Context: Legacy ETL code for example DataStage is being refactored into PySpark using Prophecy low-code no-code and available converters. Converted code is causing failures/performance issues.Skills: Spark Architecture – component...

  • Data Architect

    Found in: Talent UK 2A C2 - 2 weeks ago

    Sheffield, United Kingdom First Derivative Full time

    Are you a technologist who has a hunger to dig deep and get involved in providing first-class solutions to real-world problems?Big Data Architect - Remote UK First Derivative is a people-first company. We are powered by our ever-growing teams of specialist problem-solvers and process-evolvers. Our extensive capabilities offer agile solutions and exciting...

  • Principal Process Modelling Engineer

    2 weeks ago

    Sheffield, United Kingdom Qinecsa Solutions Full time

    Principal Technical Engineer As a principal technical engineer, you will have the opportunity to shape and drive the technical vision, and be responsible for designing, developing, and maintaining robust data-centric products and integration solutions. As a senior member of the development team, you will be expected to lead development efforts and...

  • Principal Technical Engineer

    2 weeks ago

    Sheffield, United Kingdom Qinecsa Solutions Full time

    Principal Technical Engineer As a principal technical engineer, you will have the opportunity to shape and drive the technical vision, and be responsible for designing, developing, and maintaining robust data centric products and integration solutions. As a senior member of the development team, you will be expected to lead development efforts and coach /...