Bowen's RE-Log
Hi, this is Bowen. I'm documenting my learning notes in this blog.
Conv2d - PyTorch 2D Convolutional Layer
Table of Contents Conv2d API Signature Example: Conv2d(1, 32, 3, stride=2, padding=1) Output Size Formula Derivation Worked example Common cases What Each Filter Learns References torch.nn.Conv2d applies a 2D convolution...
VAE with ELBO
Table of Contents Interpreting the ELBO Maximizing the ELBO: The Reparameterization Trick VAE Training Algorithm Summary: Overall Optimization VAE Training Algorithm in PyTorch (Gaussian Decoder) References We want to maximize...
LLM Post-Training Iterative SFT, RLHF Branching
Table of Contents Core Intuition Mathematical Foundation PPO in RLHF Why $\text{SFT}i \rightarrow \text{SFT}{i+1}$ is the Default When Starting from $\text{RL}_i$ is Fine Reward Hacking: Deep Dive Component of Insights...
Why not model.forward() but model()
Table of Contents Core Intuition Analogy Component of Insights Connections Implementation Notes Core Intuition Calling model(x) invokes nn.Module.__call__, a wrapper around forward() that runs hooks, manages state, sets up autograd,...
Monte Carlo Better than Quadrature
Table of Contents Core Intuition Mathematical Foundation Taylor’s Theorem with Remainder Proof of the Error Rate Bound in $\delta^r$ Derivation of Taylor’s Theorem with Remainder Formula Starting Point: Fundamental Theorem...
Manual Layer
Table of Contents Core Intuition Mathematical Foundation Forward Pass Backward Pass (Derivation) Key Equation Intuitive Gradient Rules Analogy Component of Insights Pitfalls Connections Implementation Notes Core Intuition A Linear Layer...
Xavier Initialization vs He Initialization
Table of Contents Core Intuition Mathematical Foundation Xavier: linear regime He: compensating for ReLU Comparison Practical Guidance Implementation Notes Connections References [Updated on 2026-08-07: Extend to Xavier Initialization vs He...
Correspondence Analysis
Table of Contents Core Intuition Example applications Archaeological example: sites vs. types Mathematical Foundation Setup Correspondence Matrix and Profiles Weighted Least Squares Formulation Generalized SVD Row and Column Profile Matrices...
PCA and SVD
Table of Contents Core Intuition Mathematical Foundation Setup PCA Assumptions Eigendecomposition SVD Matrix Version of SVD Connection Between PCA and SVD How Many Principal Components to Use Reduced Rank Approximation...