Lead Data Engineer – Dublin (Hybrid)

About the role

We are seeking a Lead Data Engineer to join a dynamic team building advanced data platforms for financial crime detection in Dublin. The role combines hands‑on engineering with leadership, designing scalable Databricks‑AWS solutions and guiding a team of engineers to deliver governed data pipelines.

Key responsibilities

• Build and optimise Spark/PySpark batch and streaming pipelines on Databricks.
• Define lake‑house architecture (Bronze/Silver/Gold), enforce schema governance and data‑quality standards.
• Design secure AWS data infrastructure using S3, Glue, Lake Formation, Lambda and Step Functions.
• Orchestrate workflows with Airflow, Databricks Workflows and Step Functions.
• Mentor engineers, lead code reviews and champion CI/CD best practices.

Required profile

• 10+ years of data‑engineering experience.
• Proven leadership and stakeholder collaboration abilities.
• Experience delivering data solutions for financial‑crime or related domains.

Required skills

• Databricks
• Apache Spark / PySpark
• Python
• SQL
• AWS (S3, Glue, Lake Formation, Lambda, Step Functions)
• Airflow
• Databricks Workflows
• CI/CD pipelines