Data Scientist – PySpark & Generative AI

About the role

We are seeking a Data Scientist with strong PySpark expertise and experience in Generative AI to join a fully remote, technology‑focused team serving the financial services sector. You will help design and deliver AI‑driven solutions that enable digital transformation and intelligent automation across large‑scale data platforms.

Key responsibilities

• Design and implement advanced Generative AI applications using large language models and intelligent agents.
• Build, optimize, and maintain high‑performance data pipelines with Python, PySpark, and Apache Spark.
• Integrate AI capabilities into scalable data environments while ensuring reliability and observability.
• Collaborate with cross‑functional, international teams in an agile setting to deliver innovative solutions.
• Participate in architectural discussions and contribute to the evolution of data and AI platforms.
• Develop production‑ready AI components with monitoring, evaluation, and RAG‑based guardrails.
• Support continuous improvement by identifying automation and optimization opportunities.
• Document technical decisions, best practices, and share knowledge within engineering teams.

Required profile

• Proven professional experience in Python and/or PySpark for data engineering, data science, or AI projects.
• Hands‑on expertise with Apache Spark and distributed data processing environments.
• Experience designing Generative AI solutions, including AI agents, RAG architectures, and LLM‑based applications.
• Understanding of AI evaluation, monitoring, observability, and production deployment best practices.
• Ability to develop scalable, high‑performance solutions in enterprise‑grade settings.

Required skills

• Python
• PySpark
• Apache Spark
• Distributed data processing
• Generative AI
• Large language models (LLM)
• AI agents
• RAG architectures
• AI evaluation methodologies
• Monitoring and observability
• Production deployment of AI systems
• Orchestration tools for AI workflows