Blog
-
Are SAO's Critic Updates Really Better?
Why a lower critic loss on the current rollout batch does not necessarily mean better value estimates for the actor.
-
Why Normalize GRPO Advantages? A Variance-Stabilizing Perspective
Connecting GRPO's standard-deviation normalization to a minimax variance-stabilizing transformation of binary success rates.
-
Token-Mean Aggregation Is Asymptotically Unbiased
How the law of large numbers explains the asymptotic unbiasedness of token-mean aggregation, up to a positive gradient scale.