Bfloat zero - #1

Open

mali-git wants to merge 16 commits into

Commits on Sep 7, 2022

Allreduce bfloat in fp32 in zero
Samuel Weinbach
committed
b0eb1c5
Tied grad reduce for bfloat
Samuel Weinbach
committed
ecb85a5

Commits on Sep 20, 2022

Correct op_builder import
rrutmann
committed
55bf9f8
Correct import from git_version_info
rrutmann
committed
f58fc1b

Commits on Oct 24, 2022

Test communication type
mali-git
committed
e794448
Merge branch 'bfloat_zero' of https://github.com/OpenGPTX/DeepSpeed into bfloat_zero
rrutmann
committed
2dfdce5

Commits on Oct 25, 2022

Test parameters' type & communication type
mali-git
committed
1466e72
Merge branch 'bfloat_zero' of https://github.com/OpenGPTX/DeepSpeed into bfloat_zero
rrutmann
committed
390562e
Test Zero stage 0 and 1
mali-git
committed
5ee97ef

Commits on Oct 26, 2022

Test _exec_reduce_tied_grads
rrutmann
committed
86d5b42
Add TiedLinearStackPipe
rrutmann
committed
c6ba9ae
Resolved merge conflicts
rrutmann
committed
2a39442

Commits on Nov 8, 2022

Initial draft of test__exec_backward_pass
rrutmann
committed
12c4315
Adapt assertion
mali-git
committed
d5d8ea1

Commits on Nov 29, 2022

Add comments
mali-git
committed
2d343ff
Add comment
mali-git
committed
256ca0b