Senior Databricks Data Engineer
Version 1 · Dublin
Job description
About the role
Version 1 is looking for an experienced Senior Data Engineer to join a team delivering data and AI solutions in a complex enterprise environment. The role covers designing and building modern data platforms, AI workflows and Generative AI solutions using Databricks and Azure technologies, including RAG pipelines, multi-agent workflows and MLOps.
Key responsibilities
- Design and build scalable ETL/ELT and AI pipelines with Databricks, PySpark, Delta Lake and SQL
- Develop and maintain Lakehouse architectures with Bronze, Silver and Gold layers
- Build and optimise Delta Live Tables, Databricks Workflows and job orchestration
- Create and maintain RAG pipelines and AI agent frameworks
- Support MLOps, including model training and inference with MLflow
- Implement data quality monitoring, governance and metadata management via Unity Catalog
- Develop event-driven integrations with Azure services and integrate Databricks with ADLS Gen2, Azure Data Factory, Azure Key Vault and Azure DevOps
- Help build feature stores and AI training datasets, and mentor engineering teams on AI technologies
Required profile
- Strong experience as a Data Engineer, AI Engineer, Machine Learning Engineer or similar
- Hands-on experience building production-grade multi-agent workflows, RAG solutions and AI agent frameworks
- Strong Python and SQL skills, and experience with Apache Spark and distributed data processing
- Experience implementing Generative AI in Databricks Workflows and Jobs, and event-driven Azure integrations
- Knowledge of data modelling and Medallion/Lakehouse architecture
- Git-based version control and CI/CD, and understanding of AI/ML pipelines
- Excellent English communication skills
Required skills
- Databricks, PySpark, Delta Lake, Unity Catalog, MLflow
- Azure Databricks, Azure Service Bus, Azure Functions, Azure Data Factory, Azure Data Lake Storage Gen2, Azure Key Vault, Azure DevOps
- Desirable: PyTorch/TensorFlow, Event Hub, Logic Apps, enterprise-scale AI platforms, mentoring experience, financial services exposure, advanced LLMOps
What we offer
- Quarterly performance-related profit share scheme
- Career progression and mentorship coaching with a quarterly career development programme
- Flexible or remote working
- Pension, private healthcare cover, life assurance, financial advice and employee discount scheme
- Gym discounts, bike-to-work, fitness classes, mindfulness workshops and an Employee Assistance Programme
- Generous holiday allowance, enhanced maternity/paternity leave and special leave policies
- Educational assistance and incentivised certifications (AWS, Microsoft, Oracle, Red Hat)
Questions fréquentes
Why are you reporting this job?
Explore further
Salaries, guides and searches in Ireland.
Salary: Senior Software Engineer - AI & ML Innovation Based on 12 job offers in IrelandApply in 30 seconds
Enter your email to apply. An account will be created automatically.
By continuing, you accept our terms of use.
Already have an account? Login
A question about this job?
Ask it here: you will get the full job summary by e-mail, right away.
Published 2 hours ago
Expires 1 month from now
4 views · 0 interested
Boost your chances
Upload your CV — we will match you with relevant openings.
Analyzing your CV...
Version 1
Dublin