Welcome to the Field Notes
A short tour of what this site is, the problems I write about, and how to get the most out of the cheatsheets.
If data were a physical thing, I’d be part plumber, part architect, and part traffic controller. This site is where I write down what I learn while keeping retail-scale pipelines honest.
No growth-hacked listicles. No “10 Spark tricks” that were outdated in 2019. Just the reasoning behind the systems I build, the mistakes that cost me a weekend, and the numbers that actually drove the decision.
What you’ll find here
Start here
- Field notes — long-form write-ups of real problems: PySpark internals, Airflow orchestration, warehouse modelling, and the cost math nobody warns you about.
- Cheatsheets — dense, copy-paste-ready references for PySpark, Airflow, SQL, and dbt. They live in the sidebar and get updated as my toolkit evolves.
How I think about data engineering
Three principles show up in almost everything I build:
- Data first, compute second, cluster last. You don’t buy a truck and then figure out what to carry — you measure the load first.
- Idempotent by default. A pipeline you can rerun safely is a pipeline you can trust at 3 AM.
- Cost is a feature. A design that can’t explain its own bill isn’t finished.
Getting around
Use the sidebar to jump between All Articles, the DE Cheatsheet dropdown, and the Archive. The activity bell in the top corner tracks the latest updates, and the search on the home page matches on titles, tags, and summaries.
Thanks for stopping by — now let’s go make some pipelines that don’t break.