Job description
What we need you to tackle
You are a strong generalist data engineer who works like a scientist: curious, precise, and comfortable with messy data problems. You have 4+ years of data engineering experience with dbt and the modern data stack, plus at least a year using coding agents as your main way of working. You explain your thinking clearly, check the data yourself, and do not rely on LLM output without validating it. Experience with web scraping, LLM-based enrichment, evals, Dagster, and English-speaking international teams is especially relevant.
Non-negotiable hard skills
- Data Processing
- Python
- Snowflake
- Data Engineering
- Artificial Intelligence
Your day-to-day
Diagnose data quality issues across the venue pipeline, including duplicates, missing fields, wrong mappings and inconsistent formats.
Form hypotheses, run experiments and evals, and present before-and-after results backed by data.
Improve existing pipelines and rebuild parts of them where it makes sense, while keeping entry and exit points intact.
Design LLM-based extraction and enrichment workflows for text and images with traceability and validation built in.
Ingest data from APIs and web sources, and maintain dbt models and Dagster workflows.
Work closely with the research team to automate more of the manual venue discovery and enrichment process.
Nice to haves
- Implements resilient ingestion from REST/GraphQL/HTML sources with pagination, auth, rate-limit handling, and idempotent loads to feed the pipeline.
- Builds and maintains reliable orchestrations with scheduling, retries, sensors, and observability to keep data flows healthy.
- Develops and maintains scrapers with stable selectors, anti-bot strategies, and drift monitoring to acquire venue data unavailable via APIs.