Commit 423118e
Document activation/aggregation behavior found by adversarial review
Add a scaling, clamping, and special-case reference table to
docs/activation.rst covering the 9 activation functions that deviate
from their canonical textbook forms. Previously the only prose in the
file was the introductory sentence "some of these functions are scaled
differently from the canonical versions" — with no indication of which
functions, what scaling factors, or what clamping ranges are used. The
new table documents:
- sigmoid, tanh, sin, softplus input scaling (5x, 2.5x, 5x, and 5x in
with 0.2x out, respectively) and ±60 input clamping
- gauss ±3.4 input clamp and -5 exponent coefficient
- exp ±60 input clamp
- log 1e-7 input floor (so non-positive inputs return log(1e-7) rather
than raise ValueError)
- inv ArithmeticError -> 0.0 fallback on division by zero or overflow
- lelu 0.005 leak coefficient explicitly noted as non-standard, with
a reference to the conventional 0.01 used by PyTorch nn.LeakyReLU
- The 9 remaining activations (relu, elu, selu, identity, clamped,
abs, hat, square, cube) listed as canonical with no scaling
Add empty-input behavior notes to all 7 aggregation function docs in
docs/module_summaries.rst. Previously these were documented as pure
math formulas (\max(x), \min(x), etc.) with no mention of what the
functions return for an empty input iterable. max, min, maxabs, median,
and mean all have explicit "if x else 0.0" guards in the source; sum
inherits Python's sum([]) = 0 behavior; product inherits reduce's 1.0
initializer. These are deliberate and address a real edge case
(orphaned nodes with no incoming connections), but the behavior was
invisible to anyone reading the docs.
Rewrite the validate_aggregation prose in docs/module_summaries.rst
from the inaccurate "takes at least one argument" to the accurate
"callable with exactly one positional argument", and document the
builtin early-return fallback that mirrors validate_activation's new
behavior. Expand the :raises: clause to enumerate the three conditions
under which InvalidAggregationFunction is raised.
Sphinx build passes: make clean html produces 18 warnings, all of
which are pre-existing in other files (academic_research.rst,
xor_example.rst, genome-interface.rst, installation.rst,
reproduction-interface.rst); none originate in activation.rst or
module_summaries.rst.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>1 parent 93d0164 commit 423118e
2 files changed
+66
-14
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
14 | 51 | | |
15 | 52 | | |
16 | 53 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
76 | 76 | | |
77 | 77 | | |
78 | 78 | | |
| 79 | + | |
| 80 | + | |
79 | 81 | | |
80 | 82 | | |
81 | 83 | | |
82 | | - | |
| 84 | + | |
83 | 85 | | |
84 | 86 | | |
85 | 87 | | |
86 | 88 | | |
87 | | - | |
| 89 | + | |
| 90 | + | |
88 | 91 | | |
89 | 92 | | |
90 | 93 | | |
91 | | - | |
| 94 | + | |
92 | 95 | | |
93 | 96 | | |
94 | 97 | | |
95 | 98 | | |
96 | | - | |
| 99 | + | |
| 100 | + | |
97 | 101 | | |
98 | 102 | | |
99 | 103 | | |
100 | | - | |
| 104 | + | |
101 | 105 | | |
102 | 106 | | |
103 | 107 | | |
104 | 108 | | |
105 | | - | |
| 109 | + | |
| 110 | + | |
106 | 111 | | |
107 | 112 | | |
108 | 113 | | |
109 | | - | |
| 114 | + | |
110 | 115 | | |
111 | 116 | | |
112 | 117 | | |
113 | 118 | | |
114 | | - | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
115 | 123 | | |
116 | 124 | | |
117 | 125 | | |
118 | | - | |
| 126 | + | |
119 | 127 | | |
120 | 128 | | |
121 | 129 | | |
122 | 130 | | |
123 | 131 | | |
124 | 132 | | |
125 | | - | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
126 | 136 | | |
127 | 137 | | |
128 | 138 | | |
129 | | - | |
| 139 | + | |
130 | 140 | | |
131 | 141 | | |
132 | 142 | | |
| |||
135 | 145 | | |
136 | 146 | | |
137 | 147 | | |
| 148 | + | |
138 | 149 | | |
139 | 150 | | |
140 | 151 | | |
141 | | - | |
| 152 | + | |
142 | 153 | | |
143 | 154 | | |
144 | 155 | | |
| |||
152 | 163 | | |
153 | 164 | | |
154 | 165 | | |
155 | | - | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
156 | 169 | | |
157 | 170 | | |
158 | 171 | | |
159 | | - | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
160 | 175 | | |
161 | 176 | | |
162 | 177 | | |
| |||
0 commit comments