TensorFlow distributed training strategies
1. TensorFlow distributed training strategies
Person: Distributed training is particularly useful for very large data sets because it becomes very difficult-- and often unrealistic-- to perform model training on only a single hardware accelerator such as a GPU. TensorFlow's distribution strategies make it easier to seamlessly scale up heavy training workloads across multiple hardware accelerators, be they GPUs or even TPUs. But in doing so, you may face challenges. For example, tf.distribute. Strategy can help with these and other potential challenges. It is a TensorFlow API to distribute training across multiple GPUs, multiple machines, or TPUs. There are four TensorFlow distributed training strategies that support data parallelism. The list includes: We'll cover each strategy in more depth in the videos that follow.2. Let's practice!
Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.