Data Engineers

Key Responsibility

You will be assigned a portfolio of client engagements where you will be expected to:

• Design and build scalable data platforms using modern cloud-native and Lakehouse architectures
• Develop and optimise data pipelines using Python, SQL, and tools such as Azure Data Factory, AWS Glue, Google Cloud Dataflow, Databricks, and dbt
• Modernise legacy data environments, migrating from on‑premises solutions to cloud-native platforms such as Microsoft Fabric, Azure Synapse Analytics, AWS Redshift, Google BigQuery, or Databricks
• Engage with clients to conceptualise data solutions aligned to their business strategy
• Support our sales team with pre‑sales activities, proof‑of‑concept deliveries, and technical proposals
• Provide technical guidance and mentorship to junior and intermediate consultants
• Lead technical reviews and contribute to consultants' growth plans
• Identify opportunities to automate manual processes, optimise data delivery, and improve infrastructure scalability
• Work with stakeholders, including executive, product, and analytics teams, to address data infrastructure needs
• Drive knowledge sharing through technical blogs, internal forums, and workshops
• Balance billable project work with team support responsibilities

Requirements

Data Engineer – Candidate Requirements

Intermediate Level

3–5 years' experience

• 3–5 years of hands‑on experience in data engineering.

• Strong proficiency in Python and/or SQL, including query optimisation.

• Experience working with both relational and non‑relational databases.

• Experience designing and building data pipelines and data models.

• Understanding and practical experience with lakehouse architectures, including the medallion pattern.

• Practical experience with at least one major cloud platform, including:

  • Microsoft Azure
  • AWS
  • Google Cloud Platform (GCP)

• Familiarity with:

  • Databricks
  • Snowflake
  • Delta Lake
  • PySpark

• Understanding of data transformation frameworks such as dbt.

• Experience with version control using Git.

• Understanding of CI/CD practices for data workflows.

• Strong analytical and problem‑solving skills.

• Ability to perform root‑cause analysis on complex data issues.

• Good communication and stakeholder engagement skills.

Senior Level

6–8+ years' experience

• 6–8+ years of hands‑on experience in data engineering.

• All intermediate-level technical requirements, together with demonstrable experience in:

  • Leading end‑to‑end data platform delivery.
  • Architecting enterprise‑grade lakehouse environments.
  • Implementing data mesh patterns.
  • Infrastructure‑as‑code using tools such as Terraform, Bicep, AWS CDK or Pulumi.
  • DevOps and CI/CD pipelines.
  • Working effectively with cross‑functional teams in a dynamic consulting environment.
  • Mentoring junior engineers.
  • Contributing to technical strategy and solution direction.

Qualifications

• Bachelor's degree in:

  • Computer Science
  • Information Systems
  • Information Technology
  • or a related field.

• Master's degree in a relevant field is advantageous.

Certifications

One or more of the following certifications would be advantageous:

• Microsoft Fabric Data Engineer Associate
• Microsoft Azure Data Engineer Associate
• Databricks Certified Data Engineer Associate
• Google Professional Data Engineer
• AWS Certified Data Engineer – Associate
• Databricks Certified Data Engineer Professional

Languages & Frameworks

• Python
• PySpark
• SQL
• dbt

Microsoft Fabric & Azure

• Microsoft Fabric Lakehouses
• Fabric Pipelines
• Fabric Semantic Models
• Direct Lake
• Azure Data Factory
• Azure Data Lake Storage Gen2
• Azure Synapse Analytics
• Azure Databricks
• Azure Event Hubs

Google Cloud Platform

• BigQuery
• Cloud Storage
• Dataflow
• Dataproc
• Pub/Sub

Amazon Web Services

• Amazon S3
• AWS Glue
• Amazon Redshift
• Amazon EMR
• Amazon Kinesis

Databricks & Data Platforms

• Databricks
• Delta Lake
• Unity Catalog
• MLflow
• Databricks Workflows

Databases

• Azure SQL
• Azure Cosmos DB
• PostgreSQL
• Snowflake
• BigQuery
• Amazon Redshift

DevOps & Infrastructure as Code

• Git
• Azure DevOps
• GitHub Actions
• Terraform
• Bicep
• AWS CDK
• CI/CD pipelines

Streaming & Messaging

• Azure Event Hubs
• Azure Stream Analytics
• Apache Kafka
• Amazon Kinesis
• Google Pub/Sub

Visualisation & Analytics

• Microsoft Power BI
• Microsoft Fabric Real-Time Dashboards
• Looker / Looker Studio
• Amazon QuickSight

#J-18808-Ljbffr