AI Data Engineer

About the role

As an AI Data Engineer you will help iCompare build a modern data and AI platform, enabling safe and effective AI‑enabled capabilities across the business. You will combine data engineering expertise with emerging AI technologies to deliver real business value.

Key responsibilities

• Design, build and optimise scalable data pipelines and AI‑ready data products for batch, streaming and event‑driven architectures.
• Develop production‑grade retrieval pipelines supporting document ingestion, embeddings, vector search, semantic retrieval and Retrieval Augmented Generation (RAG) solutions.
• Establish reusable patterns for AI enablement, including prompt orchestration, model integration, evaluation workflows and model‑ready datasets.
• Apply DataOps and MLOps practices for deployment, monitoring, testing, observability and continuous improvement.
• Champion responsible AI, governance and secure‑by‑design engineering practices to ensure reliability, scalability and compliance.

Required profile

• Strong data engineering capability with a passion for emerging AI technologies.
• Experience building scalable cloud‑based data solutions that support AI and machine‑learning use cases.
• Excellent communication skills and ability to engage with both technical and non‑technical stakeholders.

Required skills

• Databricks, Spark, PySpark, Python, SQL
• Generative AI concepts: RAG, embeddings, vector search, prompt orchestration, LLM applications
• Software engineering best practices: CI/CD, testing, infrastructure as code, version control
• DataOps, MLOps, governance, security, responsible AI practices

What we offer

• Vibrant, social community with annual celebrations and regular events
• Flexible work arrangements – hybrid model with one day remote per week or a 9‑day fortnight
• Additional leave options: ‘ME’ leave, ‘Volunteer Day’ leave, ability to purchase extra leave
• Return‑to‑work bonus for returning parents
• Quarterly award and recognition program