From 5e9b6695bd71ce7dbb2de624cbb43ecda3baf2e6 Mon Sep 17 00:00:00 2001 From: lg Date: Fri, 6 Nov 2020 13:02:43 +0000 Subject: [PATCH 1/2] split doc (wip) --- docs/api.rst | 52 +++++++++++++++++++++++++++++++++++++++++++++++++ docs/index.rst | 53 +++----------------------------------------------- 2 files changed, 55 insertions(+), 50 deletions(-) create mode 100644 docs/api.rst diff --git a/docs/api.rst b/docs/api.rst new file mode 100644 index 000000000..54fe1d3a6 --- /dev/null +++ b/docs/api.rst @@ -0,0 +1,52 @@ +KungFu APIs +=========== + +KungFu has the high-level optimizer APIs that +allows you to transparently scale out training. +It also has a low-level API that allows an easy implementation +of distributed training strategies. +The following is the public API we released so far. + +Distributed optimizers +---------------------- + +KungFu provides optimizers that implement various distributed training algorithms. +These optimizers are used for transparently scaling out the training of +`tf.train.Optimizer `_ +and `tf.keras.optimizers.Optimizer `_ + +.. automodule:: kungfu.tensorflow.optimizers + :members: + +Global variable initializers +---------------------------- + +KungFu provide various initializers to help you synchronize +the global variables of distributed training workers at +the beginning of training. These initializers are used +with ``tf.session``, ``tf.estimator``, ``tf.GradientTape`` +and ``tf.keras``, respectively. + +.. automodule:: kungfu.tensorflow.initializer + :members: + +Cluster management +------------------ + +When scaling out training, you often want to adjust +the parameters of your training program, for example, +sharding the training dataset or scaling the learning rate +of the optimizer. This can be achieved using the following +cluster management APIs. + +.. automodule:: kungfu.python + :members: + +TensorFlow operators +-------------------- + +KungFu provides TensorFlow operators to help you realise +new distributed training optimizers. + +.. automodule:: kungfu.tensorflow.ops + :members: diff --git a/docs/index.rst b/docs/index.rst index f19dc7107..5a8d9da75 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -190,58 +190,11 @@ You simply pass an extra `with_keras` flag to both KungFu optimizers and Keras callback to tell KungFu you are using Keras not TensorFlow. Here is a full Keras training example: `Keras `_ -KungFu APIs -=========== -KungFu has the high-level optimizer APIs that -allows you to transparently scale out training. -It also has a low-level API that allows an easy implementation -of distributed training strategies. -The following is the public API we released so far. +.. toctree:: + :maxdepth: 2 -Distributed optimizers ----------------------- - -KungFu provides optimizers that implement various distributed training algorithms. -These optimizers are used for transparently scaling out the training of -`tf.train.Optimizer `_ -and `tf.keras.optimizers.Optimizer `_ - -.. automodule:: kungfu.tensorflow.optimizers - :members: - -Global variable initializers ----------------------------- - -KungFu provide various initializers to help you synchronize -the global variables of distributed training workers at -the beginning of training. These initializers are used -with ``tf.session``, ``tf.estimator``, ``tf.GradientTape`` -and ``tf.keras``, respectively. - -.. automodule:: kungfu.tensorflow.initializer - :members: - -Cluster management ------------------- - -When scaling out training, you often want to adjust -the parameters of your training program, for example, -sharding the training dataset or scaling the learning rate -of the optimizer. This can be achieved using the following -cluster management APIs. - -.. automodule:: kungfu.python - :members: - -TensorFlow operators --------------------- - -KungFu provides TensorFlow operators to help you realise -new distributed training optimizers. - -.. automodule:: kungfu.tensorflow.ops - :members: + apis Indices and tables ================== From 118e06eac76974d7a3d2f49d28aa0f134499514c Mon Sep 17 00:00:00 2001 From: lg Date: Fri, 6 Nov 2020 14:17:02 +0000 Subject: [PATCH 2/2] fix build --- docs/index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/index.rst b/docs/index.rst index 5a8d9da75..c7adb1bb0 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -194,7 +194,7 @@ Here is a full Keras training example: `Keras