Senior Data Scientist - AI Model Validation

About the role

CommBank's AI & General Model Validation (AIGMV) team reviews the Group's most important machine learning and AI models across business and support units. As a Senior Data Scientist, you will independently review, challenge and validate AI use cases, assessing whether AI systems are fit for purpose, operate as intended and have their risks understood and controlled. Your analysis underpins model approvals, usage decisions and risk acceptance.

Key responsibilities

• Independently review, challenge and validate AI use cases across the Group, covering data, methodology, assumptions and outcomes
• Design validation that is proportionate to the risk and explain findings clearly to technical and non-technical stakeholders
• Work closely with model developers, product owners and business stakeholders while remaining independent, providing constructive challenge that supports safe and sustainable AI adoption
• Help develop the team's AI validation practices, tooling and standards, and share knowledge and build capability

Required profile

• Courageous mindset and a proven record of self-directed learning, keeping up with fast-changing AI techniques and risks
• Close attention to detail and a knack for spotting problems
• Strong communication skills, able to explain complex technical concepts to mixed audiences
• Experience designing, building and evaluating AI and generative AI solutions, including synthetic data generation; fine-tuning transformer models is a plus

Required skills

• Working knowledge of GenAI and agentic architecture (orchestrators, sub-agents, RAG pipelines, tool use, guardrails, multi-agent routing, memory, persona handling, LLM-as-a-Judge)
• Experience with GenAI frameworks such as LangChain, OpenAI Agents SDK, LangGraph, Pydantic, Langfuse/HoneyHive or similar
• Advanced Python, with strong GitHub, version control and CI/CD experience; SQL, R or TypeScript is a plus
• Ability to read unfamiliar codebases, reproduce evaluation pipelines, write test harnesses and inspect trace data
• Hands-on experience with AI coding agents such as Claude Code and Codex, and building sub-agents, hooks and plugins
• Working knowledge of traditional ML models such as regression, random forests and XGBoost