AI Engineer

We’re partnering with a leading Australian university undertaking a major technology transformation in 2026, with a strong focus on AI adoption, legacy modernisation, and enhanced digital experiences.

An opportunity exists for an experienced AI Engineer to help deliver an AI-powered conversational search capability within a large-scale student platform. This role will see you working on a production-grade system, transforming traditional search into a context-aware, multi-turn conversational experience.

About the Role

You’ll take ownership of the backend and retrieval pipeline, working closely to defined architecture while ensuring the system is scalable, observable, and production-ready from day one.

This is a hands-on engineering role where quality, latency, and cost are equally critical.

Key Responsibilities

RAG Pipeline Development

• Build and optimise the end-to-end retrieval pipeline (ingestion, chunking, embeddings, vector storage, retrieval, reranking)
• Implement and tune hybrid search (vector + keyword/BM25)
• Develop query understanding capabilities (rewriting, expansion, intent classification)
• Optimise performance across latency, cost, and accuracy

LLM Integration & Prompting

• Integrate LLMs into the retrieval pipeline with strong grounding and response consistency
• Design and refine prompt templates for accurate, reliable outputs
• Implement guardrails to reduce hallucinations and improve confidence signalling
• Manage model trade-offs across performance, cost, and quality

Evaluation & Testing

• Establish and track metrics (retrieval quality, response relevance, latency, cost per query)
• Build automated regression testing for pipeline changes
• Support evaluation frameworks such as RAGAS or similar
• Contribute to UAT with clear, interpretable outputs

Backend Engineering

• Design and develop APIs for frontend and system integrations
• Implement multi-turn conversation state management
• Ensure observability through logging, monitoring, and analytics
• Work closely to architecture direction, raising risks early

What Success Looks Like

• First 30 days: System understood, baseline metrics established, key improvements identified
• Mid engagement: Enhanced pipeline in staging with measurable uplift and stable APIs
• Delivery: Production-ready solution with strong performance and clean handover

Essential Experience

• Proven experience building and deploying RAG systems in production
• Strong understanding of the end-to-end retrieval stack (embeddings, vector DBs, BM25, reranking)
• Production-level Python engineering (clean, testable, maintainable code)
• Experience working within defined architecture frameworks
• Ability to work autonomously in a consulting/contract environment
• Strong focus on balancing quality, latency, and cost
• Mindset of instrumentation before optimisation

Highly Regarded

• Experience with multi-turn conversational AI systems
• Familiarity with RAG evaluation frameworks (RAGAS, TruLens, etc.)
• Exposure to enterprise knowledge platforms or higher education environments
• Experience with integration/automation tooling (e.g. n8n, Make)

If you’re looking for a change and would like to find out more about this opportunity apply now or reach out to ray.stewart@talentinternational.com for a confidential discussion