Senior Databricks Engineer | AWS Databricks

  • Australia
  • Sydney
  • Contract
  • AU$1000 - AU$1200 per day + Inclusive of Super

We are seeking a Senior Databricks Engineer for an initial three-month contract with a leading Australian financial services organisation.

The successful candidate will help mature an existing Databricks platform into an enterprise-grade capability. This is a hands-on role focused on platform scalability, automation, governance and the reduction of technical debt.

Engagement details

  • Three-month contract.
  • Senior, hands-on engineering engagement.
  • Immediate or near-term availability preferred.

Key responsibilities

  • Design and establish a governed, domain-aligned Unity Catalog structure incorporating Bronze, Silver, Gold and Reporting layers.
  • Support the migration from the organisation’s existing flat catalog structure to the new target-state architecture.
  • Re-platform existing ingestion and transformation frameworks to align with the new catalog model.
  • Develop metadata-driven, reusable and scalable data pipelines.
  • Implement RBAC and ABAC controls across all catalog layers, ensuring least-privilege and domain-based access.
  • Refresh the underlying AWS storage and security foundations, including S3 buckets, bucket policies, IAM roles and access controls.
  • Maintain and extend Terraform-based Infrastructure as Code across the Databricks and AWS environments.
  • Automate Databricks cluster provisioning and SQL Warehouse deployments, minimising manual configuration.
  • Support a phased, low-risk migration that protects critical regulatory and business reporting workloads.
  • Essential experience

Essential experience

  • Strong commercial experience engineering and supporting enterprise Databricks platforms.
  • Hands-on Databricks experience within AWS environments.
  • Demonstrated experience designing and implementing Unity Catalog architectures.
  • Strong understanding of lakehouse and medallion architecture, including Bronze, Silver and Gold data layers.
  • Experience developing metadata-driven ingestion and transformation frameworks.
  • Advanced Spark and PySpark engineering skills.
  • Strong knowledge of Databricks governance, RBAC, ABAC and least-privilege security models.
  • Practical AWS experience covering S3, bucket policies, IAM roles and secure cross-service access.

Highly desirable

  • Experience within banking, payments, superannuation or another regulated financial-services environment.
  • Knowledge of CPS 230 or other APRA-regulated reporting requirements.
  • Experience supporting financial risk, regulatory or client SLA reporting.

If you are interested, please apply now. For a confidential discussion, please reach out to Josh.costigan@talentinternational.com

Apply now

Submit your details and attach your resume below. Hint: make sure all relevant experience is included in your CV and keep your message to the hiring team short and sweet - 2000 characters or less is perfect.