Technical Lead Data Engineering

About the role

The National Heavy Vehicle Regulator is seeking a strategic, hands‑on Technical Lead for Data Engineering to guide a high‑performing team delivering smarter regulatory outcomes. You will own the quality and architecture of the NHVR Enterprise Data Platform and mentor engineers in modern data practices.

Key responsibilities

• Accountable for data pipeline quality, integrity, performance, uptime, cost, security, governance and architectural integrity on the NHVR Enterprise Data Platform.
• Own and evolve platform architecture, driving the transition to Lakeflow declarative pipelines and modern Databricks capabilities.
• Design and continuously improve a metadata‑driven ingestion and transformation framework, ensuring scalability and maintainability.
• Define, document and enforce engineering standards covering coding, Delta Lake design, testing, CI/CD and observability.
• Provide advisory guidance to senior leadership on data architecture, augmentation and manageability.
• Mentor and coach Data Engineers, lead technical planning in Scrum, and act as escalation point for complex incidents.

Required profile

• Master’s degree in IT or Computer Science (or equivalent) with at least 10 years of experience.
• Extensive experience with Databricks and Azure cloud platform, preferably with certifications.
• Proven ability to lead technical teams in Scrum/Agile environments and influence without direct authority.
• Strong communication skills to explain complex technical matters to senior non‑technical stakeholders.

Required skills

• Python, PySpark, advanced SQL (Spark SQL, T‑SQL)
• Databricks (notebooks, Jobs/Workflows, cluster configuration, performance tuning)
• Delta Lake (MERGE, upsert, schema evolution, OPTIMIZE, VACUUM, clustering)
• Unity Catalog security and governance models
• Azure DevOps, CI/CD pipelines, Git workflows
• ELT/ETL design, REST APIs, SFTP, change data capture, streaming
• Data architecture and modelling (relational, dimensional, medallion)

What we offer

• Competitive salary of $166,662.51 per annum plus superannuation.
• Hybrid working environment (up to 2 days remote).
• Monthly RDO, paid leave over Christmas, leave loading 17.5%.
• Wellbeing programs, discounted fitness, health insurance, financial services.
• Salary packaging options, discounted IT products, professional development support.