Talent.com
Placements24
Lead Data EngineerPlacements24 • Durban, KwaZulu-Natal, ZA
Search for other jobs
Lead Data Engineer

Lead Data Engineer

Placements24 • Durban, KwaZulu-Natal, ZA
16 hours ago
Job type
  • Full-time
Job description

We are seeking a highly capable Data Engineer with strong experience in scalable data platforms, large-scale scraping, LLM integrations, and agentic AI systems. This role is ideal for a hands‑on builder who enjoys solving complex data challenges and developing intelligent automation solutions that operate reliably in production environments.

You will play a key role in designing and building modern data pipelines, AI-powered workflows, and autonomous systems that can interact with tools, APIs, databases, and enterprise data sources.

Responsibilities

  • Design and build scalable batch and near-real-time data pipelines.
  • Experience specifically with Claude Code.
  • Ingest and process data from APIs, databases, files, event streams, and web sources.
  • Develop and maintain large-scale scraping and data acquisition systems.
  • Clean, validate, transform, and model data for analytics and AI applications.
  • Build LLM-powered workflows and agentic AI solutions using:
  • Structured outputs
  • Retrieval-augmented generation (RAG)
  • API and database integrations
  • Support modern data lake, warehouse, and lakehouse architectures.
  • Implement monitoring, observability, schema validation, and data quality controls.
  • Collaborate with engineering teams to deliver secure, scalable, and maintainable production systems.

Qualifications and experience

  • Strong experience with Python and/or Node.js/TypeScript.
  • Advanced SQL and relational database skills.
  • Experience with AWS or similar cloud platforms.
  • Hands‑on experience with data lakes, warehouses, or lakehouse technologies.
  • Practical experience using LLM platforms such as OpenAI, Anthropic, Gemini, Bedrock, or similar.
  • Experience building agentic workflows involving:
  • Tool use
  • Retrieval systems
  • Memory
  • Experience with scraping frameworks such as Playwright, Puppeteer, Scrapy, or Selenium.
  • Strong understanding of APIs, containers, CI/CD, testing, and secure engineering practices.

The reference number for this position is NG61354 which is a contract remote role offering a salary of up to R650 per hour salary negotiable based on experience.

#J-18808-Ljbffr

Create a job alert for this search

Lead Data Engineer • Durban, KwaZulu-Natal, ZA