AllreduceCallback

class AllreduceCallback(reduce_method, group=WORLD, backend=None)

Allreduce Callback with tensor fusion optimization.

Parameters:

• reduce_method (str) – the method to reduce gradiants.

• group (Group) – communication group.

• backend (Optional[str]) – override distributed backend in allreduce