Data Architect

2 weeks ago


Leeds UK, West Yorkshire, United Kingdom Damia Group Full time

Spark/PySpark Architect - 12 months+ -£Inside IR35- Hybrid working of 3 days on site in Leeds


My client are a Global Consultancy who are looking for a number of Spark/PySpark Architects to join them on a Long term programme. As the Spark architect, you will have the opportunity to work with one of the biggest IT landscapes in the world. You can also look forward to opportunity to collaborate with senior leadership stakeholders, guiding technical team and drive overall program objectives.


Responsibilities:

  • Working on an Enterprise scale Cloud infrastructure and Cloud Services in one of the Clouds (GCP).
  • Drive Data Integration upgrade to PySpark
  • Collaboration with multiple customer stakeholders
  • Knowledge of working with Cloud Databases
  • Excellent communication and solution presentation skills.
  • Able to analyse Spark code failures through Spark Plans and make correcting recommendations
  • Able to review PySpark and Spark SQL jobs and make performance improvement recommendations
  • Able to understand Data Frames / Resilient Distributed Data Sets and understand any memory related problems and make corrective recommendations
  • Able to monitor Spark jobs using wider tools such as Grafana to see whether there are Cluster level failures. As a Spark architect, who can demonstrate deep knowledge of how Cloudera Spark is set up and how the run time libraries are used by PySpark code.


Mandatory Skills

  • At least 12+ Years of IT Experience with Deep understanding of component understanding around Spark Data Integration (PySpark, scripting, variable setting etc.), Spark SQL, Spark Explain plans.
  • Spark SME – Be able to analyse Spark code failures through Spark Plans and make correcting recommendations.
  • To be able to traverse and explain the architecture you have been a part of and why any particular tool/technology was used.
  • Spark SME – Be able to review PySpark and Spark SQL jobs and make performance improvement recommendations.
  • Spark – SME Be able to understand Data Frames / Resilient Distributed Data Sets and understand any memory related problems and make corrective recommendations.
  • Monitoring –Spark jobs using wider tools such as Grafana to see whether there are Cluster level failures.
  • Cloudera (CDP) Spark and how the run time libraries are used by PySpark code.
  • Prophecy – High level understanding of Low-Code No-Code prophecy set up and its use to generate PySpark code.

Spark/PySpark Architect - 12 months+ -£Inside IR35- Hybrid working of 3 days on site in Leeds



  • Leeds, UK, West Yorkshire, United Kingdom Fruition Consulting Full time

    Role: Software Solutions ArchitectLocation: UK Remote (very ad-hoc international travel)Rate: Market - Outside IR35Length: Initial 6 months (lots of scope for extensions)Fruition Consulting have a new requirement for an experienced Software Solutions Architect with extensive experience designing complex AWS cloud native solutions.The Software Solutions...

  • Solutions Architect

    4 weeks ago


    Leeds, UK, West Yorkshire, United Kingdom G.Digital Full time

    AWS Solutions Architect | Up to £90,000 | Hybrid/ Yorkshire About the businessG.Digital have partnered with a Global Data Analytics Software house in the Clean Energy sector who are on the lookout for an AWS native Solution Architect to come join them in a brand new area of the business to design and deploy a brand new AWS platform. Why did we choose to...

  • Solutions Architect

    4 weeks ago


    Leeds, UK, West Yorkshire, United Kingdom G.Digital Full time

    AWS Solutions Architect | Up to £90,000 | Hybrid/ Yorkshire About the businessG.Digital have partnered with a Global Data Analytics Software house in the Clean Energy sector who are on the lookout for an AWS native Solution Architect to come join them in a brand new area of the business to design and deploy a brand new AWS platform. Why did we choose to...

  • Data Architect

    3 days ago


    Chiswick, UK, West London, United Kingdom Damia Group Full time

    **Data Architect - £650-750 per day INSIDE IR35 – 6-12 month initial contract – Chiswick (hybrid)**We are looking for a Data Architect to join a global International Health and Security Risk Management organisation on a contract basis. This is a hybrid role, with the expectation being 2-3 times per week in the Chiswick office. The Data Architect will be...

  • Spark Architect

    1 month ago


    Leeds, UK, West Yorkshire, United Kingdom PRACYVA Full time

    Spark Architect / SMEContract Role- 6 months to begin with & its extendableLocation: Leeds, UK (min 3 days onsite)Context: Legacy ETL code for example DataStage is being refactored into PySpark using Prophecy low-code no-code and available converters.Converted code is causing failures/performance issues.Skills:Spark Architecture – component understanding...

  • Spark Architect

    4 weeks ago


    Leeds, UK, West Yorkshire, United Kingdom PRACYVA Full time

    Spark Architect / SMEContract Role- 6 months to begin with & its extendableLocation: Leeds, UK (min 3 days onsite)Context: Legacy ETL code for example DataStage is being refactored into PySpark using Prophecy low-code no-code and available converters.Converted code is causing failures/performance issues.Skills:Spark Architecture – component understanding...

  • Data Engineer

    1 month ago


    Leeds, UK, West Yorkshire, United Kingdom Fruition IT Full time

    Data Engineer Leeds - on site 2x per week up to £55,000Looking to join a dynamic team in the financial sector? Fruition IT are looking for an experienced Data Engineer to join a leading company specializing in financial solutions.Working closely with the Senior Data Engineer and Architects, you will play a crucial role in their journey of migrating from...

  • Data Engineer

    4 weeks ago


    Leeds, UK, West Yorkshire, United Kingdom Fruition IT Full time

    Data Engineer Leeds - on site 2x per week up to £55,000Looking to join a dynamic team in the financial sector? Fruition IT are looking for an experienced Data Engineer to join a leading company specializing in financial solutions.Working closely with the Senior Data Engineer and Architects, you will play a crucial role in their journey of migrating from...


  • Leeds, UK, West Yorkshire, United Kingdom Wella Company Full time

    THE ROLEAs a Cloud/Infrastructure Architect, you will play a pivotal role in shaping and implementing our cloud and infrastructure strategies to support our business objectives. Collaborating closely with cross-functional teams, you will design, deploy, and optimize robust cloud-based solutions and infrastructure that drive innovation, scalability, and...

  • Azure Data Engineer

    2 weeks ago


    Leeds, UK, West Yorkshire, United Kingdom HCLTech Full time

    Job description:hands-on experience in business intelligence development or data engineering experience,Solid experience with data modelling, data warehouse design and data lake concepts and practices; Exposure working in a Microsoft Azure Data Platform environment. Exposer inAzure Data Factory Azure StorageData Ingestion and Storage including Azure Data...

  • Data Architect

    1 month ago


    Burgess Hill, UK, West Sussex, United Kingdom LTIMindtree Full time

    Hi, We are looking for Data Architect resource. Please find below the JD. Experience - 7-9 years Hybrid - 3 days working Job description:Analyse business process, coordinating with business partners to formulate architecturalstrategies for technical solutions and developing functional design to deliver business objectives.Participate in the development and...

  • Data Architect

    4 weeks ago


    Burgess Hill, UK, West Sussex, United Kingdom LTIMindtree Full time

    Hi, We are looking for Data Architect resource. Please find below the JD. Experience - 7-9 years Hybrid - 3 days working Job description:Analyse business process, coordinating with business partners to formulate architecturalstrategies for technical solutions and developing functional design to deliver business objectives.Participate in the development and...

  • Database Architect

    1 month ago


    Leeds, UK, West Yorkshire, United Kingdom HCLTech Full time

    Define Architecture standards and principals• Translation for business requirements into Data Design, technical specifications• Data migration, strategy, mapping / lineage, quality, standards across multiple projects and solutions• Work closely with the development team to design and implement data solutions• Being able to recommend the best place to...

  • Technology Architect

    1 month ago


    Leeds, UK, West Yorkshire, United Kingdom Infosys Full time

    Location: Leeds, UKSkill: ReactJs & Spring; Java API/Microservices Must have Skills:.Full stack architecture – ReactJs & Spring; Java API/Microservices.Containerization (Kubernetes) and Workflow orchestration (Temporal).Strong skills in distributed architecture – local and cross site resiliency and failover.Achieving scalability on Apache Nifi based data...

  • Data Architect

    4 weeks ago


    Sheffield, UK, South Yorkshire, United Kingdom BJSS Full time

    Do you want to work with cutting edge data and analytics technologies to help our clients unlock their data potential and value?Do you want to lead and mentor teams of expert technologists on exciting data projects?Do you want to work in an environment that embraces your drive to learn?About UsWe’re an award-winning innovative tech consultancy - a team of...

  • Solutions Architect

    1 month ago


    York, UK, North Yorkshire, United Kingdom La Fosse Full time

    Solution ArchitectSolution / Architecture / Digital / Transformation / AWS / Customer La Fosse have partnered with a renowned client in the transport industry, who are seeking to recruit a Solution Architect to work across their strategic digital transformation programme. This transformation covers, AWS, ML, AI, Ops systems, customer contact centre etc....

  • Solutions Architect

    4 weeks ago


    York, UK, North Yorkshire, United Kingdom La Fosse Full time

    Solution ArchitectSolution / Architecture / Digital / Transformation / AWS / Customer La Fosse have partnered with a renowned client in the transport industry, who are seeking to recruit a Solution Architect to work across their strategic digital transformation programme. This transformation covers, AWS, ML, AI, Ops systems, customer contact centre...


  • Birmingham, UK, West Midlands, United Kingdom VIQU IT Recruitment Full time

    Lead Salesforce Technical ArchitectBirmingham - HybridUp to £110,000VIQU have partnered with one of the UK’s largest providers of business software and services, serving a diverse clientele of over 20,000 global customers. The organisation stands as a titan in the UK's business software and services sector, catering to an expansive, international...

  • AWS Data Architect

    1 month ago


    Cheshire East, UK, North West England, United Kingdom Wipro Full time

    About Us:Wipro Limited (NYSE: WIT, BSE: 507685, NSE: WIPRO) is a leading global information technology, consulting, and business process services company. We harness the power of cognitive computing, hyper-automation, robotics, cloud, analytics and emerging technologies to help our clients adapt to the digital world and make them successful. A company...

  • AWS Data Architect

    4 weeks ago


    Cheshire East, UK, North West England, United Kingdom Wipro Full time

    About Us:Wipro Limited (NYSE: WIT, BSE: 507685, NSE: WIPRO) is a leading global information technology, consulting, and business process services company. We harness the power of cognitive computing, hyper-automation, robotics, cloud, analytics and emerging technologies to help our clients adapt to the digital world and make them successful. A company...