Data Engineer - South Africa

Data Engineer - Data Integrations / Big Data Engineer

We are seeking a Data Integrations / Big Data Engineer with experience in Apache NiFi, Kafka, and enterprise systems to design and implement secure, scalable, and real-time data pipelines for a high-profile client implementation project. The ideal candidate will have expertise in managing structured and semi-structured data, integrating legacy and modern systems, and automating data flows across cloud and on-prem environments.

Key Responsibilities

• Design, develop, and maintain data pipelines for ingesting and transforming data from various sources (including APIs, flat files, DBs).
• Integrate messaging systems like Kafka and RabbitMQ into the pipeline for event streaming and system decoupling.
• Work closely with Business Analysts and BPEs to translate client requirements into technical data flows.
• Implement and manage connectors for REST APIs, and file-based batch processes.
• Ensure data lineage, auditability, and performance tuning of NiFi flows.
• Deploy and monitor data flows in distributed environments using Zookeeper, Kafka, and ElasticSearch for logging and observability.
• Strong experience in designing and implementing data integration solutions for real-time streaming data.

Required Skills

Nice to Have

Other Requirements

Candidates must be eligible to obtain and maintain security clearance at the highest level, in accordance with applicable national security regulations.

This role requires full-time, on-site presence at the client’s premises located in Pretoria. Remote or hybrid work arrangements are not applicable.

#J-18808-Ljbffr