Data Engineer

Data Engineer
12 months initial contract; with possible extension
Adelaide Based Position

Our client is looking for a highly proficient and dedicated data engineer professional who thrives in a fast-paced, supportive environment to assist with the development of a new data-driven, flagship IT system with high visibility in the sector.

Key Competencies:

Minimum of 10+ years’ experience as a Data Engineer designing, developing and deploying domain-specific data solutions with high volume, complex disparate datasets on cloud platforms, preferably Microsoft Azure.
Demonstrated experience designing production, operational data solutions that underpin business critical applications.
Experience developing data pipelines that meet availability, latency, and reliability requirements for business-critical processes.
Proven ability to independently identify and translate complex requirements from a variety of sources (e.g. functional and non-functional requirements, user stories, etc.) into technical data solution designs.
Proven experience implementing data quality controls, validation rules, reconciliation processes, and exception handling across different layers.
Experience designing auditability, traceability, and reprocessing mechanisms to support operational assurance and compliance.
Experience implementing data security controls, including role-based access control (RBAC).
Experience implementing monitoring, logging, and alerting for data pipelines and data services to ensure operational visibility and rapid issue resolution.
Experience managing schema evolution, backward compatibility, and change impacts across multiple environments (development, test, and production).
Proven experience designing, implementing and maintaining scalable ETL/ELT pipelines with complex transformations across Azure Data Factory, Databricks and SQL services that includes ingestion data from disparate datasets and sources (e.g. Databricks, SQL, API) and with various file formats (e.g. CSV, JSON, Excel, Parquet).
Advanced SQL, Python, Apache Spark, Databricks and Azure service engineering skills.
Extensive understanding of data lake and lake warehouse protocols, schema evolution and data-quality frameworks.
Experience creating data models, including those associated with software/application design, administration functionality, data process (e.g. data mastery/MDM, data lake) and presentation in reports and dashboards.
Experience with Azure DevOps (e.g. repositories, branching strategies, CI/CD, etc.).
Understanding of Australian government data governance, security and compliance standards, policies and industry best practice.
Experience with lineage mapping (source to target).
Experience developing and maintaining dbt models that follow engineering best practice.
Experience collaborating with multidisciplinary internal and external teams, including technical staff (e.g. developers, data engineers, testers, etc.), product owners, and senior leadership to inform design decisions and to validate development outcomes.
Experience working in agile, product-led delivery teams, while not essential is preferred.
Demonstrated experience and ability to set and manage one’s own work activities to ensure that timelines are met and outputs are fit-for-purpose and of a high-quality.
Communicates orally and in writing in a manner that is clear, fluent and holds the audience’s attention, and prepares and delivers information in a logical, sequential and succinct manner.
Ability and willingness to work at a rapid pace in a supporting environment with no existing system to refer to and evolving requirements.
Integrity, honesty, fairness, impartiality and commitment to values-based leadership.
Relevant qualifications preferred (e.g. Computer Science, Data Engineering, etc.).

Apply now or reach out to Ivan Aureus at 0480 806 152.