Working Path

ML Research Readiness

A serious sequence for building ML research foundations. Five phases move from probability and linear algebra through learning theory, optimization, deep learning, and current research questions.

This path is intentionally still in progress. Treat it as a foundations map, then use the diagnostics to choose a starting point.

Find your gaps first Take the diagnostic

1/5

Phase 1: Foundations assumed cold

The prerequisite layer. If these are fuzzy, later optimization and generalization arguments become memorized slogans.

Expectation, variance, momentsCore language for risk and estimation Probability distributionsKnow the Gaussian, Bernoulli, Exponential by heart Bayesian estimationPosteriors, priors, MAP, and generative modeling Eigenvalues & SVDPCA, spectral methods, conditioning Gradient descentBatch, stochastic, mini-batch: convergence tradeoffs

2/5

Phase 2: Learning theory classics

The "why do models generalize" thread. Expect precise statements of VC, Rademacher, PAC, and uniform convergence, not verbal summaries.

Bias-variance tradeoffState the decomposition exactly ERM frameworkWhy does minimizing training loss work?VC dimensionKnow the shattering argument Rademacher complexityData-dependent generalization bound PAC learningPrecise notion of learnability Concentration inequalitiesHoeffding, Bernstein, McDiarmid Uniform convergenceThe bridge from concentration to generalization

3/5

Phase 3: Optimization and training

Practical questions with theoretical answers. Can you reason about convergence rates and failure modes?

Convex optimizationStrong convexity, smoothness, convergence rates SGD convergenceVariance, step size schedules, strong convexity vs non-convex AdamWhen does it beat SGD? When does it fail?Batch normalizationWhat does it actually do? Internal covariate shift is not the answer.RegularizationL1 vs L2, early stopping, dropout as regularizer BackpropagationDerive from scratch on a 2-layer MLP

4/5

Phase 4: Modern deep learning

What research teams care about right now: transformer internals, generalization puzzles, and scaling behavior.

Transformer architectureDraw it; explain every block; why residual stream?AttentionQK^T/sqrt(d): why the sqrt? What about softmax temperature?Double descentClassical bias-variance is incomplete Implicit biasWhy does SGD find generalizing solutions?Neural tangent kernelInfinite-width limit; connection to kernels Scaling lawsKaplan vs Chinchilla; compute-optimal training

5/5

Phase 5: Research frontier

Use this as a map of open technical areas after the foundations are stable.

RLHFBradley-Terry, PPO vs DPO, reward hacking GrokkingDelayed generalization; what's the mechanism?Induction headsIn-context learning circuit; why two layers?Mechanistic interpretabilityWhat are the open problems?Sparse autoencodersSuperposition hypothesis; dictionary learning Constitutional AIRLAIF; self-critique loops

Representative questions

A sample of questions this path should make answerable. Each links to the page that carries the exact statement or derivation.

Strategy

Breadth before depth. Know every Phase 1-3 topic well enough to state the theorem and sketch the proof. Later questions usually expose weak assumptions.
Pick one frontier area. You cannot cover everything in Phase 5. Pick scaling, interpretability, or alignment and go deep.
Expect failure-mode questions. "When does X fail?" is the most common follow-up. Every page here has a FailureMode section for a reason.
Start with the gap finder. Pick the topic you want to reach. It walks prerequisites backward and gives you a reading list.