[WIP] Make FSDPv1 only perform cat() during last microbatch backward() within FlattenParamsWrapper - #1178

[WIP] Make FSDPv1 only perform cat() during last microbatch backward() within FlattenParamsWrapper#1178

chrisxcai wants to merge 11 commits into

ngoyal_changes_for_pp_fp8facebookresearch/fairscale:ngoyal_changes_for_pp_fp8from

chriscai_ngoyal_changes_for_pp_fp8facebookresearch/fairscale:chriscai_ngoyal_changes_for_pp_fp8Copy head branch name to clipboard