Data Engineer – Web Scraping

About the role

This position is based in Ireland and focuses on building scalable web‑scraping solutions and data pipelines that feed business‑critical insights. You will work closely with analysts, engineers and cross‑functional teams to deliver reliable datasets for strategic decision‑making.

Key responsibilities

• Design, develop and maintain automated web scrapers for structured and unstructured sources.
• Collaborate with analysts and stakeholders to translate data requirements into tailored solutions.
• Clean, transform, validate and manipulate large datasets using Python and Pandas.
• Build and maintain data ingestion pipelines into databases or data warehouses.
• Schedule, monitor and optimise scraping workflows with Apache Airflow.
• Implement quality‑control checks to ensure data integrity and availability.
• Investigate and resolve pipeline issues and production incidents.
• Enhance internal tools, automation frameworks and platform capabilities.
• Partner with engineering teams to create scalable, maintainable data processing workflows.

Required profile

• Bachelor's or Master's degree in Computer Science or a related technical discipline.
• 2‑4 years of professional software development experience.
• Strong programming skills in Python and solid SQL/database knowledge.
• Advanced experience with the Pandas library for data cleaning and transformation.
• Hands‑on experience with web technologies (HTML, JavaScript, APIs) and large‑scale data processing.
• Proactive problem‑solving mindset and ability to work independently.

Required skills

• Python
• SQL
• Pandas
• HTML
• JavaScript
• APIs
• Selenium
• Scrapy
• XPath
• Fiddler
• Postman
• Apache Airflow