launcher

class launcher(*args, **kwargs)

Decorator for launching multiple processes in single-machine multi-gpu training.

Parameters:

• func – the function you want to launch in distributed mode.

• n_gpus – how many devices each node.

• world_size – how many devices totally.

• rank_start – start number for rank.

• master_ip – ip address for master node (where the rank 0 is).

• port – server port for distributed server.

• backend – set default collective communication backend.