This job offer is not available in your country.
AI Operations Specialist
Africonology SolutionsJohannesburg, Gauteng, South Africa16 days ago
Job descriptionImplement automated monitoring and alerting systems for AI performance. Define SLOs and error budgets for AI features. Testing & Quality Assurance Execute manual test cases for exploratory and non-automatable scenarios. Identify, log, and track bugs to closure using Jira, Azure DevOps, or equivalent tools. Ensure adherence to QA methodologies, tools, and best practices. AI & Bot Operations Conduct offline / online evaluations and A / B tests for prompts, models, and policies. Track and troubleshoot production issues (latency, escalations, fallback errors). Design and orchestrate conversations using Voiceflow and related tools. Observability, Security & Compliance Implement guardrails, safety red-teaming, and prompt-injection defenses. Manage PII handling, content safety filters, and data loss prevention (DLP). Document incidents, perform RCA reporting, and ensure compliance with data privacy and security policies. Collaboration & Continuous Improvement Participate in incident on-call rotations, maintain runbooks, and conduct disaster recovery (DR) tests. Recommend improvements to enhance system resilience, reduce downtime, and optimize costs. Bachelor's degree in Computer Science, Engineering, or related field (or equivalent experience). 3+ years of experience in operations, DevOps, or AI / ML support roles. Proven track record managing large, complex, multi-stakeholder projects. Strong knowledge of Machine Learning practices (model monitoring, retraining, pipelines). Familiarity with conversational AI platforms (Amazon Lex, Salesforce Einstein Bots, ElevenLabs). Integration experience with Amazon Connect, Genesys Cloud CX, NICE CXone, or similar. Proficiency with test automation tools (Selenium, Playwright, Cypress, Appium, or equivalent). Experience with API testing tools (Postman, RestAssured, Karate). Strong scripting and automation skills (Python, Bash, CI / CD pipelines). C1 English proficiency. Experience with performance / load testing tools (JMeter, Locust, Gatling). Knowledge of cloud platforms (AWS, Azure, GCP). Familiarity with Git or other version control systems. QA certification (ISTQB or equivalent).
AI Operations Specialist
Responsible for ensuring the stability, reliability, and performance of AI systems in production. This role blends Machine Learning Operations (MLOps), Bot Operations, and Quality Assurance (QA) practices, supporting both backend models and customer-facing bots.
The specialist will manage deployments, monitor system health, conduct testing, and validate quality to guarantee seamless AI operations and a consistent customer experience.
Key Responsibilities
- System Monitoring & Reliability
Monitor AI systems to ensure uptime, performance, and error-free operation.
Design, develop, and maintain automated test scripts for web, mobile, and API testing.
Manage version control, deployment, and release cycles of AI models and bots.
Work with observability / evaluation tools (Langfuse, Arize / Phoenix, W&B, Prometheus, Grafana).
Collaborate with developers, data scientists, and business teams to resolve operational issues.
Requirements
Nice to Have
#J-18808-Ljbffr
Create a job alert for this search
Operation Specialist • Johannesburg, Gauteng, South Africa
Related jobs
Join Tether and Shape the Future of Digital Finance.At Tether, we’re not just building products, we’re pioneering a global financial revolution.
Our cutting-edge solutions empower businesses—from ex...Show moreLast updated: 19 days ago
Monday to Friday, 10 : 00 AM – 6 : 00 PM CST (Slightly flexible).Standard PTO policy as per company guidelines.Competitive, based on experience.
Pearl works with the top 1% of candidates from around the...Show moreLast updated: 25 days ago
%20Ltd)
Search Engine Marketing Specialist (SEM)– POS25237.Our client is seeking a highly motivated and experienced.Search Engine Marketing Specialist.
The successful candidate will manage and optimis...Show moreLast updated: 20 days ago
This opportunity is only for candidates currently residing in the specified country.Your location may affect eligibility and rates.
Please submit your resume in English and indicate your level of En...Show moreLast updated: 19 days ago
Please submit your resume in English and indicate your level of English.The Toloka Annotators connects individuals with Generative AI projects from leading tech innovators.Our mission is to unlock ...Show moreLast updated: 9 days ago
Boldr is the first global B-Corp dedicated to delivering world-class Client experiences while creating access to dignified, meaningful work in communities worldwide.
We are a global team united by o...Show moreLast updated: 21 days ago
InfyStrat is seeking an experienced AI Platform Architect to join our innovative team.In this role, you will be responsible for the design, architecture, and implementation of AI platforms that dri...Show moreLast updated: 30+ days ago
Senior Specialist : Insights & Analytics.Conduct exploratory and in-depth data analytics using SQL, Python and Spark.Complete data quality validation to identify deviations from trends and gener...Show moreLast updated: 30+ days ago
SEO / Google Ads Specialist
Talent SamSandton, GP, ZA Our Clients company is a dynamic fast-growing agency dedicated to elevating brands through innovative, culturally resonant marketing.
They partner with clients across industries to create strategies...Show moreLast updated: 30+ days ago
Senior AI Research Engineer, Model Inference (100% Remote)
Tether Operations LimitedJohannesburg, GP, ZARemote
Executive Operations Manager (US-Based AI and Automation Firm)
PearlJohannesburg, Gauteng, ZARemote
Quick Apply
SEO Specialist
RM Staffing B.V.Johannesburg, GP, ZA We are seeking a skilled and data-driven.In this role, you will be responsible for optimizing our website and content for search engines to increase organic visibility, traffic, and conversions.You...Show moreLast updated: 30+ days ago
- Promoted
Senior Business Intelligence Developer
PartnerUp (Pty) LtdJohannesburg, South Africa You will work in the IT solutions team within an Insure (Short-term Administration) company.You will be responsible for analysing, designing and building BI solutions that enable operations and bus...Show moreLast updated: 1 day ago
- Promoted
Principal AI Technology & Innovation Specialist
NTT Ltd.Johannesburg, ZA Job title : Principal AI Technology & Innovation Specialist.Job Location : Gauteng, Johannesburg.The Principal AI Technology & Innovation Specialist is a highly skilled subject matter expert with d...Show moreLast updated: 17 days ago
- Promoted
AI Technology and Innovation Engineer
NTT Ltd.Johannesburg, ZA Job title : AI Technology and Innovation Engineer.Job Location : Gauteng, Johannesburg.The AI Innovation Engineer is responsible for defining and delivering AI technology solutions that meet enterp...Show moreLast updated: 17 days ago
Search Engine Marketing Specialist (SEM)
EmporiumEdenvale, Ekurhuleni Metro, South AfricaQuick Apply
Freelance AI Red Team Engineer
MindriftJohannesburg, Gauteng, ZARemote
Quick Apply
- Promoted
AI Solution Architect
IQbusinessJohannesburg, ZA Job title : AI Solution Architect.Job Location : Gauteng, Johannesburg.We are seeking an accomplished AI Solutions Architect with a proven track record in engaging senior executives and designing s...Show moreLast updated: 30+ days ago
Freelance Data Annotation Specialist - AI Trainer
Toloka AnnotatorsJohannesburg, Gauteng, ZARemote
Quick Apply
- Promoted
Artificial Intelligence Specialist
GMI AdvisoryMidrand, Gauteng, South Africa Preffered Requirements : Associate or Bachelor’s degree in Artificial Intelligence or related field.At least 3-5 years of experience in designing and developing AI software.Strong knowledge of AI al...Show moreLast updated: 29 days ago
- Promoted
Operations Excellence Analyst
Betway AfricaJohannesburg, ZA Job title : Operations Excellence Analyst.Job Location : Gauteng, Johannesburg.As an Operations Excellence Analyst you’ll be identifying inefficiencies, driving process improvements, and enabling s...Show moreLast updated: 6 days ago
- Promoted
Senior Manager : Modelling & AI
NedbankJohannesburg, ZA Job title : Senior Manager : Modelling & AI.Job Location : Gauteng, Johannesburg.To lead and grow a high-performing team focused on advanced Machine Learning (ML) modelling and artificial intelligen...Show moreLast updated: 26 days ago
Senior Revenue Operations Specialist
BoldrJohannesburg, Gauteng, ZARemote
Quick Apply
AI Platform Architecture
InfyStratJohannesburg, Gauteng, ZAQuick Apply
- Promoted
AI Engineer
FirstRand Corporate CentreJohannesburg, ZA Job Location : Gauteng, Johannesburg.The ALMA (Asset Liability Management Analytics) team in Group Treasury is responsible for maintaining and enhancing the ALMA application, as well as providing s...Show moreLast updated: 10 days ago
- Promoted
AI Data Scientist
Network RecruitmentJohannesburg, South Africa Youll be part of a forward-thinking team thats not just talking about AI - theyre using it to drive real business transformation.
Here, your ideas wont gather dust in a presentation deck.Theyll be d...Show moreLast updated: 30+ days ago
Specialist : Insights and Analytics
Optimal Growth TechnologiesMidrand, South AfricaQuick Apply
- Promoted
- New!
AI Transformation Practice Lead – Global
NTT Ltd.Johannesburg, ZA Job title : AI Transformation Practice Lead – Global.Job Location : Gauteng, Johannesburg.AI Academy Development & Strategy – Lead the redesign, expansion, and governance of the AI Academy as NTT D...Show moreLast updated: 18 hours ago