Senior Data Engineer (Remote)

Overview

At Circana, we value continuous learning, feedback, and flexibility. We are a global company committed to inclusivity and belonging, celebrating diverse backgrounds and viewpoints. Circana is certified by Great Place To Work.

We are seeking a skilled and motivated Data Engineer to join a growing global team. You will design, build, and maintain robust data pipelines and infrastructure on the Azure cloud platform, leveraging PySpark, Apache Spark, and Apache Airflow to process and orchestrate large-scale data workloads with a focus on data quality, efficiency, and scalability.

Responsibilities

• ETL / ELT Pipeline Development : design, develop, and optimize scalable ETL / ELT pipelines using Python, PySpark, and Apache Airflow; implement batch and real-time data processing with Apache Spark; ensure data quality, governance, and security throughout the data lifecycle.
• Cloud Data Engineering : manage and optimize Azure-based data processing infrastructure for cost-effectiveness; implement and maintain CI / CD pipelines for data workflows to ensure reliable deployments.
• Big Data & Analytics : develop and optimize large-scale data processing pipelines with Apache Spark and PySpark; apply data partitioning, caching, and performance tuning; work with structured and unstructured data to support analytics and machine learning initiatives.
• Workflow Orchestration (Airflow) : design and maintain DAGs to automate complex data workflows; monitor, troubleshoot, and optimize job execution and dependencies.
• Team Leadership & Collaboration : provide technical guidance and mentorship to a team of data engineers in India; foster best practices for coding standards, version control, and documentation.

Requirements

Desirable

Circana Behaviours

Location

Bracknell

Seniority level

Mid-Senior level

Employment type

Full-time

Job function

Information Technology

Industries

Market Research

#J-18808-Ljbffr