Understanding deep learning through the lens of algebraic geometry.
About This Project
My first experiments with Singular Learning Theory, Local Learning Coefficient (LLC), grokking, and developmental interpretability. Main focus on curriculum learning and finetuning in SLT context.
README
This repository contains experiments exploring Singular Learning Theory (SLT), a mathematical framework for understanding deep learning through algebraic geometry and statistical learning theory.
Key Topics Explored
- Local Learning Coefficient (LLC) - A measure of model complexity in SLT
- Grokking - The phenomenon where models suddenly generalize after memorization
- Developmental Interpretability - Understanding how models develop capabilities over training
- Curriculum Learning - Training with progressively harder examples