The normal way in Pytorch to use multiple GPUs is to use DDP (Dynamic Data Parallel). In Pytorch documentation they have a very good tutorials. There is a certain boiler plate code we should use. We have to invoke in a special way. It will produce several versions of the process and each version will use a different GPU. It will not change the code, it will be only wrapper around the code.
Just by having a code base we normally can't say if it is using multiple GPUs. But we can search for words "rank", "world_size" and "ddp".
Please look at this project to get an idea about how one might use DDP
https://github.com/k2-fsa/icefall/search?q=ddp
Comments
Post a Comment