10 · Foundations — MOC

growing#moc#foundations

Up: plan

Go deep on basics, not 2026 fads. The fundamentals haven't changed in decades; the frameworks will. Learn the thing under the thing.

The bar for every note here: Can I derive/visualize it from memory and compute a tiny example by hand? — not "have I read about it." Each note ends with a by-hand exercise that meets that bar.

Anti-fad rule: be wary of any topic that's been hot for < 6 months. Learn the 40-year-old idea underneath it first.

The ten

Note The bar / what to be able to do
Cross-entropy and KL divergence Compute both by hand for a 3-outcome distribution
SVD and low-rank structure Visualize it; relate to PCA, embeddings, matrix factorization
Probability for sequences Joint/conditional/marginal, Bayes, MLE vs MAP
Markov chains and HMMs Transition matrices, stationary dist, forward/Viterbi
Point processes Poisson (homogeneous + inhomogeneous), Hawkes self-excitation
Heavy-tailed distributions Log-normal, Pareto, Zipf; why clickstreams have fat tails
Information theory basics Entropy, mutual information, perplexity
Evaluation theory Precision/recall/AUC/log-loss, calibration, proper scoring rules
Causal inference primer Confounding, potential outcomes, why prediction ≠ causation
Optimization & gradients SGD; the ideas behind policy gradients (not the framework du jour)

The texts behind them

Verified starter bibliography — which chapter backs which note — in Foundations reading list. The sequenced path across the whole vault is in Reading roadmap.

How these connect downstream

linked from