Senior Data Engineer (Remote)

Overview

At Circana we are fueled by our passion for continuous learning and growth we seek and share feedback freely and we celebrate victories both big and small in an environment that is flexible and accommodating to our work and personal lives. Were a global company dedicated to fostering inclusivity and belonging. We value and celebrate the unique experiences cultures and viewpoints that each individual brings. By embracing a wide range of backgrounds skills expertise and beyond we create a stronger more innovative environment for our employees clients and communities. Circana is proud to be Certified by Great Place To Work. This prestigious award is based entirely on what current employees say about their experience working at Circana.

Learn more at .

What will you be doing

We are seeking a skilled and motivated Data Engineer to join a growing global this role you will be responsible for designing building and maintaining robust data pipelines and infrastructure on the Azure cloud platform. You will leverage your expertise in PySpark Apache Spark and Apache Airflow to process and orchestrate large-scale data workloads ensuring data quality efficiency and scalability. If you have a passion for data engineering and a desire to make a significant impact we encourage you to apply!

Job Responsibilities

• ETL / ELT Pipeline Development : Design, develop and optimize efficient and scalable ETL / ELT pipelines using Python, PySpark and Apache Airflow.
• Implement batch and real-time data processing solutions using Apache Spark.
• Ensure data quality governance and security throughout the data lifecycle.
• Cloud Data Engineering : Manage and optimize cloud infrastructure (Azure) for data processing workloads with a focus on cost-effectiveness.
• Implement and maintain CI / CD pipelines for data workflows to ensure smooth and reliable deployments.
• Big Data & Analytics : Develop and optimize large-scale data processing pipelines using Apache Spark and PySpark.
• Implement data partitioning caching and performance tuning techniques to enhance Spark-based workloads.
• Work with diverse data formats (structured and unstructured) to support advanced analytics and machine learning initiatives.
• Workflow Orchestration (Airflow) : Design and maintain DAGs (Directed Acyclic Graphs) in Apache Airflow to automate complex data workflows.
• Monitor troubleshoot and optimize job execution and dependencies within Airflow.
• Team Leadership & Collaboration : Provide technical guidance and mentorship to a team of data engineers in India.
• Foster a collaborative environment and promote best practices for coding standards version control and documentation.

Requirements

Desirable

Circana Behaviours

As well as the technical skills experience and attributes that are required for the role our shared behaviours sit at the core of our organization. Therefore we always look for people who can continuously champion these behaviours throughout the business within their day-to-day role :

Location

This position can be located in the following area(s) : Bracknell

#LI-KM1

Required Experience

Senior IC

Key Skills

Apache Hive,S3,Hadoop,Redshift,Spark,AWS,Apache Pig,NoSQL,Big Data,Data Warehouse,Kafka,Scala

Employment Type

Full-Time

Department / Functional Area

Data Engineering

Experience

years

Vacancy

#J-18808-Ljbffr