Principal Research Scientist – Evaluations

About the role

Canva is looking for a Principal Research Scientist to lead the evaluation of its generative AI models. You will define and own the measurement framework that turns subjective design judgments into reliable, actionable metrics across the company.

Key responsibilities

• Define and drive the evaluation strategy for design, image, video, audio and agentic workflows.
• Validate metrics against human preference, quantify judge bias and prevent benchmark saturation.
• Bridge evaluation results with user experience and product impact, ensuring metrics guide model training and deployment.
• Build a shared evaluation layer across global teams to ensure consistent measurement.
• Establish evaluation gates for launch decisions and diagnose anomalous results in production.

Required profile

• Proven track record creating evaluation frameworks for generative AI at scale.
• Experience linking evaluation metrics to downstream business or user outcomes.
• Strong understanding of multimodal generative models (diffusion, transformers, VLMs, MLLMs).
• Background in reward modelling, preference learning or human‑feedback alignment methods.
• Ability to set technical direction across distributed teams.

Required skills

• Diffusion models
• Transformers
• Vision‑language models (VLMs)
• Multimodal large language models (MLLMs)
• Reward modelling and preference learning
• RLHF / RLAIF techniques