Talent.com
Vivo Energy
Principal Data Engineer (Databricks)Vivo Energy • Cape Town, Western Cape, ZA
Principal Data Engineer (Databricks)

Principal Data Engineer (Databricks)

Vivo Energy • Cape Town, Western Cape, ZA
30+ days ago
Job description
Role Overview Reporting to the Data & Analytics Lead, the Principal Data Engineer is a hands-on role, combining the building and optimisation of production data pipelines with the technical leadership needed to raise engineering standards and develop the people around them. The role functions as a key interface between a) business stakeholders and department heads, b) the wider data and analytics team, and c) owners of source systems and data in Finance and IT. Key Responsibilities - Designing and building robust, production-grade data pipelines on Databricks, making appropriate use of current platform capabilities such as Spark Declarative Pipelines (SDP) - Working “under the hood” to optimise the platform for performance and cost — tuning compute, jobs and queries, managing storage and table layout, and keeping platform spend under control - Defining and maintaining consistent business metrics and a semantic layer (for example, using Metric Views) so reporting is built on trusted, reusable definitions - Integrating data from core business systems, including SAP S/4HANA, and other sources, using tools such as Fivetran (both SaaS connectors and HVR) - Setting and upholding engineering standards across the team — code quality, testing, documentation, CI/CD and data governance - Coaching and mentoring junior and mid-level engineers, reviewing their work and helping them develop - Partnering with business stakeholders to understand their needs, shape practical solutions and ensure the platform delivers genuine value - Engaging data owners in Finance and IT to ensure data is well understood, valid and fit for purpose, and initiating data quality improvements where required Requirements Skills & Experience Experience - At least 3 years of hands-on, daily Databricks experience, covering both pipeline development and under-the-hood performance and cost optimisation - Up to date with recent platform developments, such as Spark Declarative Pipelines (SDP) and Metric Views - Strong proficiency in programming languages commonly used in data engineering, such as Python, SQL and Spark - Advanced experience with data manipulation, data modelling, database design and query optimization - Experience managing or coaching junior developers - A track record of managing and influencing business stakeholders - Experience working with SAP S/4HANA data sets, Fivetran (including both SaaS connectors and HVR), Power BI semantic modelling would be beneficial Key Competencies - Combining deep, hands-on engineering skill with sound judgement about cost, performance and long-term maintainability - Coaching, mentoring and raising the capability of less experienced engineers - Collaborating, communicating confidently and influencing business stakeholders - Breaking down complex technical concepts and explaining them simply to non-technical audiences - Staying current with a fast-moving platform and bringing new capabilities into everyday practice - Taking ownership and driving work independently, from concept through to production Important to note: This role requires full-time office-based attendance, five days per week. To apply Qualified candidates to apply by uploading a cover letter and a recent resume by close on business 31 July 2026.
Create a job alert for this search

Principal Data Engineer (Databricks) • Cape Town, Western Cape, ZA

Similar jobs

Investment Technology Process Engineer (SQL) (CPT)

Datafin RecruitmentCape Town, ZA

A renowned Independent Asset Management Firm seeks a results-driven Investment Technology Process Engineer with strong SQL skills and a minimum of 3 years’ experience within Financial Services or A... Show more

 • Promoted

Data Engineer

PrescientCape Town, ZA

This Data Engineer is a senior individual contributor responsible for leading the design, delivery, and operational support of complex, business‑critical data pipelines integrating investment and f... Show more

 • Promoted

Senior Data Engineer

iOCOCape Town, ZA

We are looking for a highly skilled, hands-on Senior Data Engineer with deep expertise in SQL Server, T-SQL development, data modelling, and database performance optimization.This role is primarily... Show more

 • Promoted

Data Engineer (Python, SQL, PostgreSQL, Node.js) – (In Office) CPT

Datafin RecruitmentCape Town, ZA

An Investment company is searching for an innovative Data Engineer to join their growing team in Durbanville, Cape Town.If you're passionate about building scalable data solutions, solving complex ... Show more

 • Promoted

Data Engineer (Python, PySpark, Azure/AWS) (CPT)

Datafin RecruitmentCape Town, ZA

Own data products from ingestion through to trusted consumption, ensuring data is reliable, accessible, and fit for purpose.Design, build, and operate scalable ELT and ETL pipelines across structur... Show more

 • Promoted

Data Engineer

PBT GroupCape Town, Western Cape, ZA

PBT Group is seeking an experienced Data Engineer to join a high-performing data team responsible for building and maintaining modern, scalable data platforms that enable enterprise-wide analytics ... Show more

 • Promoted

Data Integration Engineer

Salt South AfricaCape Town, ZA

Our client is looking for a Data Integration Engineer who is passionate about backend engineering, distributed systems, and cloud-native architectures.This is a hands-on role where you’ll design an... Show more

 • Promoted

Senior Java Engineer (Data Engineering) (CPT Hybrid)

Datafin RecruitmentCape Town, ZA

WORK on cutting-edge Data Engineering projects of a US-based growing premium Software Product and Consulting Services company seeking a Senior Java Engineer with strong Data Engineering.You will be... Show more

 • Promoted

Data Engineer

CollinsonCape Town, Western Cape, South Africa
Quick Apply

Collinson is the global, privately-owned company dedicated to helping the world to travel with ease and confidence.We use our expertise and products to craft customer experiences which enable some ... Show more

Data Scientist

IGrow Wealth InvestmentsCape Town, ZA

IGrow Wealth Investments Group of Companies, proudly positioned as South Africa’s #1 Investment company, is seeking a talented and driven Data Scientist to join our innovative and growing team base... Show more

 • Promoted

Investment Technology Process Engineer (SQL) (CPT)

Datafin IT RecruitmentCape Town, Western Cape, ZA

ENVIRONMENT: A renowned Independent Asset Management Firm seeks a results-driven Investment Technology Process Engineer with strong SQL skills and a minimum of 3 years' experience within Financial... Show more

 • Promoted

Solutions Architect - Databricks

Decision IncCape Town, Western Cape, South Africa
Quick Apply

The key focus for the senior data/AI architect is to perform planning aligned to key AI solutions, build and participate in the architecture capability building, perform AI architecture and design,... Show more

Senior BI Engineer

AiRCape Town, Western Cape, South Africa
Quick Apply

Help us empower millions across Africa by building the data foundation for a leading fintech platform.You'll design and implement BI solutions that drive critical business decisions.BI Engineering ... Show more

Data Principal Engineer

takealot.comCape Town, ZA

Our data and analytics platform powers the operational backbone of Takealot Group, from logistics and supply chain to distribution centre analytics and data monetisation products.It's grown fast, a... Show more

 • Promoted

Data Engineer (Python, PySpark, Azure/AWS) (CPT)

Datafin IT RecruitmentCape Town, ZA

ENVIRONMENT: PLAY a key role in delivering trusted data products that enable investment, operational, and client-facing decision-making across an innovative Investment Management firm seeking a sk... Show more

 • Promoted

Data Engineer

Salt South AfricaCape Town, ZA

Our client is seeking a Data Engineer to build and scale the data infrastructure behind a fast-growing investment business.You will play a key role in designing systems that enable reporting, autom... Show more

 • Promoted

Data Engineer (Mid–Senior)

Hire ResolveCape Town, ZA

A Data & AI Intelligence consultancy company is looking for a highly motivated and experienced Data Engineer to join a collaborative team of Product Owners, Data Engineers, and Machine Learning... Show more

 • Promoted

Data Scientist

Datonomy SolutionsCape Town, Western Cape, ZA

Data Scientist / AI Developer Cape Town | Contract Opportunity We are looking for a skilled Data Scientist / AI Developer to join a high-performing retail technology environment focused on cuttin... Show more