Data Analysis & Web Scraping

Using AI for data pipelines, WRDS access, EDGAR filings, web scraping, and analytical workflows.

Summaries

Data Analysis Workflows

• Data Analysis for Economists — Goldsmith-Pinkham's Census data analysis demo (Markus Academy Ep. 162-2)
• Large Datasets and Structured Databases — 70 GB HMDA → DuckDB + Parquet; metadata as context engineering
• Large Datasets (Video) — Live Markus Academy walkthrough of HMDA pipeline (Ep. 162-4)
• From Empty Folder to Figure — Sub-agents, Kieran Healy styling, Claude Code vs Cowork

Web Scraping & EDGAR

• Web Scraping for Economists — SEC EDGAR scraping with plan mode (Markus Academy Ep. 162-3)
• EDGAR Filings to Structured Database — Seven lessons from building an EDGAR pipeline

Text Classification & NLP

• PNAS Replication Part 1 — Cunningham replicating Card et al. immigration rhetoric with OpenAI Batch API
• PNAS Replication Part 2 — Results: 69% agreement, $11 cost, polarization finding robust

Research Tools & Templates

• Claude WRDS Tools — Orlowski's toolkit for querying CRSP, Compustat, TAQ via Claude Code
• ChernyCode Template — Boris Cherny's productivity template: memory, skills, subagents