Data Engineer - (AI & Databricks)
We are seeking an experienced Data Engineer with a passion for AI, cloud technologies, and modern data engineering practices. This role is ideal for someone who enjoys solving complex integration challenges, leveraging AI to improve engineering outcomes, and building scalable data solutions within AWS.
The successful candidate will work on a range of cloud integration initiatives, including integrating Model Context Protocol (MCP) servers with AWS services, while helping shape next-generation AI-enabled data platforms.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines and integration solutions.
- Lead and deliver MCP server integrations with AWS services and cloud infrastructure.
- Develop and optimise data solutions using Databricks and modern data engineering best practices.
- Leverage AI tools and techniques to improve engineering productivity, automation, and solution delivery.
- Build and optimise data ingestion, transformation, and orchestration workflows.
- Collaborate with software engineers, solution architects, and business stakeholders to deliver robust, scalable data solutions.
- Ensure data quality, security, governance, and performance across data platforms.
- Support cloud migration and modernisation initiatives where required.
- Document technical solutions and contribute to engineering standards and best practices.
Required Skills & Experience
- 5+ years’ experience as a Data Engineer or in a similar data-focused engineering role.
- Strong background in modern data engineering principles and best practices.
- Mandatory hands-on experience with Databricks, including building and optimising data pipelines and working with the Databricks Lakehouse Platform.
- Strong experience with AWS services such as S3, Lambda, Glue, Athena, Redshift, ECS, IAM, API Gateway, or similar.
- Experience building cloud-native data integration solutions.
- Knowledge of Model Context Protocol (MCP) server integrations, or the ability to quickly learn and implement them.
- Strong experience with Python, SQL, Spark/PySpark, and data orchestration tools.
- Solid understanding of APIs, microservices, and modern integration patterns.
- Excellent problem-solving skills and the ability to work independently in a fast-paced environment.
Desirable Skills
- Experience using AI-enabled development tools such as GitHub Copilot, ChatGPT, Claude, Amazon Q, or similar.
- Experience with Infrastructure as Code (Terraform or CloudFormation).
- Knowledge of container technologies including Docker and Kubernetes.
- Experience with CI/CD pipelines and DevOps practices.
- Exposure to data governance, security, and compliance frameworks.
What We’re Looking For
We’re looking for someone who:
- Is passionate about AI and continuously explores how it can improve engineering outcomes.
- Has a strong foundation in data engineering, Databricks, and AWS cloud technologies.
- Enjoys learning new technologies and solving complex integration challenges.
- Can work autonomously while collaborating effectively with cross-functional technical teams.
- Brings a proactive, innovative, and delivery-focused mindset.
Engagement
- Contract: 6 Months
- Start Date: ASAP
- Location: Hybrid/Flexible (subject to project requirements)
- Rate: Up to over $1,000 per day (including super)
Apply Now
If you’re a hands-on Data Engineer who thrives on solving complex technical challenges and wants to work at the forefront of AI, cloud, and modern data engineering apply now or reach out to ray.stewart@talentinternational.com