We are looking for a motivated and technically skilled Data Engineer who is eager to learn and grow. This role involves working with modern data platforms and development practices to design, build, and maintain scalable data solutions.
Responsibilities
- Build and optimize data pipelines and workflows on Databricks and Spark.
- Contribute to the design and implementation of end-to-end data flows and CI / CD processes.
- Collaborate with cross-functional teams to deliver reliable, scalable, and high-quality data solutions.
- Continuously expand your knowledge of modern data engineering practices.
Required Qualifications
Core Skills : Strong communication and collaboration abilities.Attention to detail with a focus on data quality.Experience in end-to-end data flow design and implementation.Familiarity with agile ways of working.Growth Mindset. Demonstrated eagerness to learn and adapt, supported by examples of applying new skills in practice.Technical Expertise : Proven experience with big data tools, particularly Databricks.Strong background in cluster computing using Apache Spark.Experience developing and maintaining CI / CD pipelines.Proficiency in Python with mandatory PySpark experience.Beneficial Experience
Prior work as a backend developer, showcasing strong software engineering fundamentals.Previous experience as a data engineer.Familiarity with Azure cloud infrastructure and services.What You’ll Get
Being South Africa’s largest company of its kind – IDM is continuously growing, with no signs of stopping. One of our core values is ‘growing people’ and we take that very seriously. We have internal learning opportunities to better our people, and leaders who are experts in their fields and freely impart knowledge and are open to new ideas. You can speak to anyone here. Your rank doesn’t matter. You are listened to.
Our culture is one of collaboration, diversity, and helpfulness. That goes for our people internally and how we deal with our clients. At IDM, you can play a part in educating and assisting real South Africans get out of tough financial situations. You can make a difference with the work you do. Our people believe this too.
Other key benefits
We support mental and physical health with Healthy Together campaigns throughout the year.We offer various rewards and awards as tokens of recognition for hard work.We have a vibrant office with easy access to stores and outlets.Company benefits – like a half day off on your birthday, medical aid, retirement annuity, and further study opportunities.There is potential for growth across or upwards in the business.#J-18808-Ljbffr