Senior AI Engineer – Databricks / GenAI (Healthcare)
- Full-time
Role Overview
Decision Inc. is seeking a Senior AI Engineer to develop a Generative AI (GenAI) reporting solution for a leading private hospital group. The solution will use Large Language Models (LLMs) to transform patient and clinical data stored in Databricks into accurate, clinician-ready reports. The successful candidate will be responsible for translating clinical reporting requirements into governed data structures, designing LLM-powered reporting workflows, implementing retrieval-augmented generation (RAG), establishing evaluation and safety controls, and deploying the solution into production.
Key Responsibilities
- Translate clinical reporting requirements into the required data points and build governed tables in Databricks.
- Design prompts, templates and orchestration workflows to generate structured narrative reports.
- Build retrieval capabilities over clinical guidelines and policy documents where required.
- Implement guardrails to ensure generated statements are grounded in source data and sensitive patient information remains protected.
- Develop LLM evaluation frameworks with clinical stakeholders, including test datasets, automated scoring and regression testing.
- Deploy the solution into production with monitoring, audit trails and cost controls.
- Demonstrate progress to clinical stakeholders during each sprint and incorporate feedback into ongoing improvements.
Requirements
- Experience in AI/ML engineering, including at least one LLM solution delivered into production.
- Strong Databricks experience, including: Unity Catalog, Delta Lake, Databricks Workflows, MLflow, Mosaic AI Model Serving, Vector Search, Production-standard Python, PySpark and SQL skills.
- Prompt engineering and orchestration experience using LangChain, LangGraph, DSPy or equivalent frameworks.
- Experience generating structured outputs from LLMs.
- Hands-on experience designing, tuning and evaluating retrieval-augmented generation (RAG) solutions.
- Knowledge of LLM evaluation methods, including golden datasets, groundedness scoring, LLM-as-judge and regression testing.
- Experience implementing guardrails, hallucination controls and sensitive personal data protection.
Preferred / Advantageous Requirements
- Healthcare data experience, including admissions, clinical coding (ICD-10), case management or funder interactions.
- Working knowledge of POPIA and South African health data governance.
- Azure experience, including Azure OpenAI, Key Vault and private networking.
- Databricks Generative AI Engineer or Machine Learning Professional certification.
Core Skills and Search Keywords
Senior AI Engineer, Generative AI, GenAI, LLM, Databricks, Unity Catalog, Delta Lake, MLflow, Mosaic AI, Model Serving, Vector Search, Python, PySpark, SQL, RAG, Retrieval-Augmented Generation, LangChain, LangGraph, DSPy, Prompt Engineering, LLM Evaluation, Healthcare Data, Clinical Reporting, POPIA, Azure OpenAI.
#J-18808-Ljbffr