Skip to content

Commit 9678254

Browse files
committed
Add doc for sparse=True
1 parent 2c0760d commit 9678254

File tree

1 file changed

+21
-0
lines changed

1 file changed

+21
-0
lines changed

docs/source/guide/guide_part_i.rst

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -172,6 +172,27 @@ To create a simple all-to-all connection with a weight and bias:
172172
pipeline=[weight, bias]
173173
)
174174
175+
Feature values (e.g., weights, biases) can be represented as sparse tensors for memory efficiency.
176+
To enable sparse tensor support:
177+
178+
1. Set the sparse=True parameter when initializing the feature.
179+
2. If the value tensor does not include a batch dimension, explicitly specify the batch_size parameter.
180+
181+
Example 1: Batch dimension included in the value tensor (first axis):
182+
183+
.. code-block:: python
184+
185+
weights = Weight(name='weight_feature', value=torch.rand(2, 100, 1000), sparse=True) # Batch size = 2
186+
bias = Bias(name='bias_feature', value=torch.rand(2, 100, 1000), sparse=True)
187+
188+
189+
Example 2: Batch dimension specified via batch_size parameter (no batch axis in value):
190+
191+
.. code-block:: python
192+
193+
weights = Weight(name='weight_feature', value=torch.rand(100, 1000), sparse=True, batch_size=2)
194+
bias = Bias(name='bias_feature', value=torch.rand(100, 1000), sparse=True, batch_size=2)
195+
175196
Specifying monitors
176197
*******************
177198

0 commit comments

Comments
 (0)