More transformer improvements#183

Open

coryMosaicML wants to merge 15 commits into

mosaicml:mainfrom

coryMosaicML:mu-param

Commits on Nov 26, 2024

Activation norm callback and some refactor
corystephenson-db
committed

Commits on Nov 27, 2024

Commits on Nov 28, 2024

Commits on Dec 1, 2024

Set attention scale
corystephenson-db
committed

Commits on Dec 2, 2024

Encode a la mode
corystephenson-db
committed

Commits on Dec 4, 2024