Data Platform Engineer

OverviewAt Octopus we've developed a data platform used by our businesses worldwide. The platform enables self-service data analytics and automates data processing workflows, from ETL to ML training and prediction. The data platform team works on initiatives from natural language understanding of customer communications to processing billions of smart meter readings daily to support customised and market-leading smart energy tariffs. We are looking for an Engineer to join the team to help build and maintain our platform, pipelines and reporting tools. We are passionate about building great technology to change how customers use energy for the good of the planet and this is a fantastic opportunity to work on data problems that move us closer to Net Zero and support the energy transition. We employ software engineering best practices to design, test, and deploy our data platform and services. What you’ll do Build and optimise the platform used by the data team to transform raw data into business insights Scope and deliver new data engineering projects in collaboration with business stakeholders Develop and deploy AI tools to support growing AI requirements and use cases Research and implement new tooling, whether in-house or sourced externally Maintain and expand our existing data infrastructure and tools Consult the wider business on Engineering best practices and stay up to date with industry trends Support the internationalisation of our data infrastructure as we continue to grow globally What you’ll have Primarily, a passion for writing high quality code and building lean processes Essential: Python AWS SQL (Spark/Postgres preferable) Areas for Growth: Airflow Experience using CLI tools Docker/EKS Spark DBT Circle CI The projects will be varied and we’re looking for someone who can work autonomously and proactively to scope problems and deliver pragmatic solutions Our Tech Stack Python as our main programming language Circle CI for continuous deployment Databricks for our data lake platform Parquet and Delta formats on S3 for data lake storage Terraform on Spacelift for infrastructure definition and deployment Kubernetes for data services and task orchestration Airflow for job scheduling and tracking Postgres / Aurora for relational databases Spark for data processing dbt for data modelling SparkSQL for analytics Streamlit for data applications Why you’ll love it here Salary is discussed on a call with a recruiter to match your experience and the correct level; there is flexibility and we focus on finding the right fit Octopus Energy Group has a unique culture with autonomy, co-owners, and projects that break new ground We were named best company to work for in 2022 and appear on lists for great places to work; we celebrate our culture and leadership Visit our UK perks hub – Octopus Employee Benefits Our process usually takes up to 4 weeks, but we’ll flex around what works for you. You’ll chat with our recruitment team and your Recruiter will guide you through the stages. If you have questions, email hiring@octoenergy.com. If this sounds like you, we’d love to hear from you. Are you ready for a career with us? We aim to provide you with the tools and environment to unleash your potential. If you require accommodations or have a unique preference, let us know and we’ll tailor the interview process where possible. Studies have shown that some groups of people, like women, are less likely to apply unless they meet 100% of the requirements. If you like one of our jobs, we encourage you to apply. We are an equal opportunity employer and do not discriminate on protected attributes. We commit to providing equal opportunities, an inclusive work environment, and fairness for everyone. #J-18808-Ljbffr