Senior Data Engineer

About the role

The Senior Data Engineer will design, build and evolve a cloud‑native data platform that powers business intelligence, advanced analytics, machine learning and generative AI across iSelect and Compare the Market.

Key responsibilities

• Design, develop and support scalable cloud data warehouses, lakehouses and AI‑enabled platforms.
• Build, optimise and maintain batch, streaming and near‑real‑time pipelines using Spark, Kafka, Kinesis and cloud storage.
• Implement DataOps, SecOps, CI/CD, IaC and observability practices to ensure reliable, secure data delivery.
• Provide AI‑ready datasets, vector search and RAG architectures for ML/Gen‑AI initiatives.
• Collaborate with business and technology stakeholders to govern data quality, security and compliance.

Required profile

• 7+ years in data engineering, analytics or related fields.
• 5+ years designing cloud‑based data platforms on AWS, Azure or GCP.
• Proven technical leadership, stakeholder engagement and mentoring experience.

Required skills

• Cloud platforms: AWS, Azure, GCP.
• Data platforms: Databricks, Snowflake, Fabric, Synapse, Redshift, BigQuery.
• Processing frameworks: Apache Spark, PySpark, Kafka, Kinesis, Event Hub.
• Languages: Python, SQL, Scala, Java, R, TypeScript, Julia.
• Data modelling: dimensional modelling, Data Vault, metadata management.
• Engineering practices: DataOps, CI/CD, IaC, DevOps, automated testing, observability.

What we offer

• Vibrant social community with annual celebrations and family fun days.
• Flexible work arrangements – hybrid model with one day remote per week or a 9‑day fortnight.
• Additional leave options including ‘ME’ leave, Volunteer Day, and the ability to purchase extra leave.
• Free coffee on Mondays, Return‑to‑Work bonus for new parents, and quarterly award & recognition program.