Talent.com
Rain
LLM Systems Engineer/Prompt ArchitectRain • Cape Town, South Africa
Search for other jobs
LLM Systems Engineer/Prompt Architect

LLM Systems Engineer/Prompt Architect

Rain • Cape Town, South Africa
5 days ago
Job description


SUMMARY:
purpose
We are seeking an LLM Systems Engineer to design, implement, and optimise our AI-powered customer service platform. This role requires expertise in prompt engineering, business logic translation, and production AI systems. You will work with our team to build multi-agent conversational AI systems that handle complex customer interactions across billing, technical support, and sales domains.
AI should be a force multiplier here. Use it to work faster and maintain high standards — but evaluate its output critically. We want engineers who apply judgment to amplify these tools, not avoid them or accept them blindly.

key responsibilities

  • Design and maintain modular prompt architectures with clear separation of personas, rules, workflows, and tools.
  • Translate complex business rules into executable AI behaviours and decision trees.
  • Optimise prompts for accuracy, consistency, and token efficiency; implement RAG patterns for dynamic context assembly.
  • Build production-grade AI systems using Python, FastAPI, Redis, and PostgreSQL.
  • Design tool/function-calling integrations with backend APIs; implement caching, session management, and state persistence.
  • Ensure observability through comprehensive logging, metrics, and tracing.
  • Create behaviour-driven test plans for conversational AI; conduct regression testing and coordinate A/B testing.
  • Debug inconsistent AI behaviours using conversation logs and tool execution traces.
  • Write clear technical documentation; maintain structured, markdown-based knowledge bases and runbooks.

requirements

  • Python expertise: 3+ years with FastAPI, async/await, decorators, type hints.
  • Production experience with GPT-4, Claude, or similar LLMs via API.
  • Advanced prompt engineering — context windows, token optimisation, tool-calling patterns.
  • Strong system design fundamentals (Strategy, Registry, Factory, Decorator patterns).
  • Experience with Redis (caching, sessions) and PostgreSQL.
  • RESTful API design and WebSocket communication.
  • Ability to translate complex business rules into AI workflows.
  • Experience designing conversational flows with appropriate tone, empathy, and escalation logic.
  • Systematic approach to testing emergent AI behaviour, not just code correctness.
  • Experience deploying and monitoring AI systems in production.
  • Exceptional, clear technical writing for multiple audiences.
  • Comfortable with ambiguity and hypothesis-driven iteration.

Preferred experience:

  • LangChain, LlamaIndex, or similar LLM frameworks.
  • Workflow automation platforms (n8n, Zapier).
  • Vector databases for semantic search.
  • Background in customer service, telecom, fintech, or subscription billing.
  • Compliance awareness (GDPR, audit trails, data security).
  • Voice UX design experience.

You will thrive in this role if:

  • You use AI as leverage on your judgment, not a substitute for it.


POSITION INFO:
technology stack Python and FastAPI · Redis and PostgreSQL · GPT-4 \/ Claude via API, RAG, and tool calling · production observability (logging, metrics, tracing) · AI integrated into the engineering workflow.

Create a job alert for this search

LLM Systems Engineer/Prompt Architect • Cape Town, South Africa