Cloud Engineer (Contract)

Responsibilities

• Architect, engineer and maintain robust scalable and fault tolerant data pipelines using technologies like Ab Initio, Python/Scala, Apache spark, Microsoft Fabric, Databricks for ingestion, transformation and delivery of structured, semi structured and unstructured data.
• Ensure efficient performance tuned extraction and loading of data into the enterprise Data Warehouse and Data Lake focusing on high availability, cost efficiency and reliability.
• Implement modern Data Ops practices including automation, testing and orchestration for both batch and low latency streaming data workflows.
• Collaborate with Software Engineers to design and implement consumption APIs and services that enable secure real time data access for business applications.
• Architect and manage the robust, hybrid cloud data platform utilising Azure Services (Microsoft Fabric, Ab initio, synapse, Databricks, SAS) and potentially on‑premises technologies (DB2 warehouse, Netezza, Denodo, Ab Initio).
• Establish and manage comprehensive observability for all data infrastructure components such as databases, data lakes, and data warehouses to meet strict service level objectives and aligned with architectural standards.
• Collaborate with Information Security, CISO, and Data Governance to enforce data security, policies and implement masking/anonymisation techniques to ensure compliance with data privacy regulations (POPIA).
• Lead the technical evaluation, proof of concept and the successful integration of cutting‑edge data engineering technologies and tools to enhance platform maturity and competitive advantage.
• Embed data quality as code by implementing automated, unit and end to end data validation, reconciliation, and auditing frameworks within the CI/CD data pipelines.
• Design and maintain the automated capture and maintenance of technical and operational metadata ensuring complete automated data lineage within the enterprise metadata hub (Ab Initio).
• Act as a technical guardian of the data integrity partnering with Data Governance to define and enforce master data management and data quality standards across all data products.
• Proactively manage data quality through regular data profiling, advanced issue analysis, and swift documented root cause resolution.
• Serve as trusted technical partner, collaborating with business stakeholders, Data Scientists, Analysts, and Architects to translate requirements into scalable data solutions.
• Drive the full data lifecycle of data products and end-to-end ownership of data engineering initiatives, ensuring timely delivery and alignment with business objectives.
• Provide Level 2/3 support for complex data issues, coordinating with cross‑functional teams to resolve incidents, ensuring a minimal mean time to resolution and clear communication.
• Contribute to sprint planning, backlog refinement, and agile delivery processes.
• Drive continuous optimisation of data engineering practices, standards, and processes to improve efficiency and performance.
• Mentor and guide junior engineers, providing coaching and technical support to build capability.
• Stay abreast of emerging technologies and industry trends to enhance data engineering maturity.
• Contribute to innovation initiatives that align with the bank's data-driven strategy.

#J-18808-Ljbffr