Current jobs related to Spark Architect - Cardiff, Cardiff - Damia Group LTD

  • Data Architect

    1 week ago

    Cardiff, Cardiff, United Kingdom Exalto Consulting Full time

    Job Title: Data Engineer - DatabricksJob Description:Exalto Consulting is seeking a skilled Data Engineer to work on a contract basis for a client. The role will involve working x1 days a week on site in Swansea and the rest remotely. The contract is initially 6 months and pays £525 per day, inside IR35.Key Skills and Must-Haves: Databricks (ideally on...

  • Cloud and Infrastructure Consultant

    2 weeks ago

    Cardiff, United Kingdom ARO Full time

    Job Title: Cloud and Infrastructure ConsultantLocation: UK based– Hybrid – with UK TravelSalary: Up to £65,000 + Car Allowance (£6000)Contract type: Permanent #LI-Hybrid About the roleOur Cloud and Infrastructure Consultant will be responsible for the design and successful delivery of both on premise and cloud based infrastructure projects. The purpose...

Spark Architect

3 months ago

Cardiff, Cardiff, United Kingdom Damia Group LTD Full time

Spark Architect - Leeds 3-days Onsite - 12-Month Contract - circa - (Apply online only) for Inside IR35

Exciting opportunity for an experience Spark Architect to join a large Global Consultancy, you will be working to refactor Legacy ETL code using low-code no-code and available converters. The successful candidate will have extensive experience with Spark Data Integration, Spark SQL, Spark Explain Plans, etc.

Location: Leeds, 3 days onsite.

Responsibilities:

  • Working on an Enterprise Scale Cloud Infrastructure and Cloud Services in one of the Clouds (GCP).
  • Drive Data Integration upgrade to PySpark.

Must Have Key Skills:

  • Deep understanding of Spark Data Integration (PySpark, scripting, variable setting, etc), Spark SQL, Spark Explain plans.
  • Spark SME - Experience analysing Spark code failures through Spark Plans and make the correcting recommendations.
  • Spark SME - Experience reviewing PySpark and Spark SQL jobs and making performance improvements.
  • Deep understanding of Data Frames / Resilient Distributed Data Sets.
  • Monitoring -Spark jobs using wider tools such as Grafana to see whether there are Cluster level failures.
  • Deep understanding of Cloudera (CDP) Spark and how the run time libraries are used by PySpark code.
  • High level understanding of Low-Code No-Code prophecy set up and its use to generate PySpark code.

Spark Architect - Leeds 3-days Onsite - 12-Month Contract - circa - (Apply online only) for Inside IR35

Damia Group Limited acts as an employment agency for permanent recruitment and employment business for the supply of temporary workers. By applying for this job you accept our Data Protection Policy which can be found on our website.

Please note that no terminology in this advert is intended to discriminate on the grounds of a person's gender, marital status, race, religion, colour, age, disability or sexual orientation. Every candidate will be assessed only in accordance with their merits, qualifications and ability to perform the duties of the job.

Damia Group is acting as an Employment Business in relation to this vacancy and in accordance to Conduct Regulations 2003.