Databricks Data Engineer Consultant

About the role

Accenture is looking for experienced Databricks Data Engineers to join its Data & AI practice in Australia. You will design, develop and maintain scalable data pipelines on Azure Databricks, supporting advanced analytics and data science initiatives.

Key responsibilities

• Design, develop and optimise data ingestion pipelines using Azure Data Factory, Databricks and Apache Spark.
• Build and maintain Databricks notebooks with PySpark for data preparation, transformation and enrichment.
• Collaborate with Data Scientists to deliver clean, structured datasets and implement data quality checks.
• Integrate data from structured, semi‑structured and unstructured sources while ensuring security, governance and banking compliance.
• Optimise Spark jobs for performance and cost‑efficiency in Azure and participate in code reviews and agile ceremonies.

Required profile

• Strong analytical and problem‑solving abilities.
• Excellent communication and stakeholder management skills.
• Ability to work collaboratively in cross‑functional, Scrum/Agile teams.

Required skills

• Azure Databricks and Apache Spark (PySpark).
• Azure Data Factory, Azure Synapse, Azure Blob Storage, Delta Lake.
• Python and/or Scala for data processing.
• CI/CD pipelines using Azure DevOps, Git and Terraform (preferred).
• SQL, NoSQL and data lake architecture knowledge.