Multiple GPU example limited to one GPU

[9cpluss]

tutorials/beginner_source/blitz/data_parallel_tutorial.py

Line 60 in 646c8b6

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

Isn't this line limiting the example to one GPU no matter how many GPUs are available?

cc @sekyondaMeta @svekars @carljparker @NicolasHug @kit1980 @subramen

[svekars]

Yes, if index is removed, it will be multiple GPU:
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

[callistusndemo]

/assigntome

[prithviraj-maurya]

/assigntome

[github-actions]

The issue is already assigned. Please pick an opened and unnasigned issue with the docathon-h2-2023 label.

[malfet]

Yes, if index is removed, it will be multiple GPU: device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

Sorry, can you point me to some sort of documentation that confirms that? Key component of the tutorial is use of nn.DataParallel, which puts them on mutliple GPUs, index of the device is irrelevant afterwards, as long as model is running on GPU. One can control how many GPUs are used by passing device_ids to nn.DataParallel constructor, otherwise all available ones will be used all the time.

[svekars]

Yes, you're right, then I guess we can just close this one.