Data Engineer – AWS Kinesis & Databricks

About the role

Accenture is seeking a hands‑on Data Engineer to design, build and maintain real‑time data pipelines on AWS. You will work with Amazon Kinesis and Databricks to move high‑volume streaming data into a Lakehouse architecture.

Key responsibilities

• Build and support real‑time pipelines using Amazon Kinesis Data Streams and Kinesis Data Firehose.
• Develop streaming solutions in Databricks with Spark Structured Streaming and Delta tables.
• Optimize pipelines for performance, latency, reliability and cost, and implement monitoring with AWS CloudWatch.
• Collaborate with batch data engineers to ensure seamless integration between streaming and batch workflows.
• Troubleshoot pipeline issues and contribute to continuous improvement.

Required profile

• Australian citizenship with an active security clearance.
• Hands‑on production experience with Amazon Kinesis (essential).
• Strong Python and SQL coding skills.
• Experience with Apache Spark Structured Streaming and Databricks, including Delta Lake.
• Good knowledge of core AWS services such as IAM, S3 and CloudWatch.

Required skills

• Amazon Kinesis Data Streams
• Kinesis Data Firehose
• Databricks
• Spark Structured Streaming
• Delta Lake
• Python
• SQL
• AWS IAM
• AWS S3
• AWS CloudWatch
• Kafka (beneficial)

What we offer

• 18 weeks paid parental leave
• Long & short‑term career break opportunities
• Structured career development program
• Local and international career opportunities
• Family Inclusive Workplace™ certification
• Flexible work arrangements supporting health and wellbeing