11# Geometric Entropy Lab
22
3+ > ** A continuous analogue of the Erdős distinct-distance problem.**
4+ > Points on a manifold are arranged to extremize the Shannon entropy of their
5+ > pairwise-distance distribution, producing highly symmetric, extremal
6+ > configurations on spheres, tori, cubes, and arbitrary STL meshes.
7+
38An interactive browser-based demo for optimizing point distributions on various 3D geometries using ** Spherical Gram
49Matrix Entropy** and ** TensorFlow.js** .
510
@@ -12,6 +17,21 @@ Open `index.html` in a modern browser — no build step required.
1217Points are placed on a chosen geometry (sphere, torus, cube, etc.) and optimized by maximizing or minimizing a Shannon
1318entropy derived from a Gaussian kernel density estimate over pairwise distances.
1419
20+ ### The Erdős Connection
21+
22+ The classical Erdős distinct-distance problem asks: how many * distinct* pairwise
23+ distances must $n$ points in the plane determine? It's a combinatorial counting
24+ problem with a discrete answer.
25+
26+ This lab plays the * continuous* version of the same game. Instead of counting
27+ distinct distances, we treat the multiset of pairwise distances as a probability
28+ distribution (via a Gaussian kernel) and maximize its Shannon entropy. The
29+ result is the most "distance-diverse" configuration the manifold admits — a
30+ continuous extremizer in place of a discrete counting bound.
31+
32+ Maximizing entropy spreads points apart; minimizing concentrates them; matching
33+ a target value lets you dial in a specific level of distance diversity.
34+
1535### Core Math
1636
1737```
@@ -117,6 +137,63 @@ Available variables: `rho` (Nx1), `p` (Nx1x3), `q` (1xNx3), `D` (NxN dist²), `t
117137| ** QQN** | Quasi-Quasi-Newton — faster convergence on smooth losses |
118138| ** L-BFGS** | Limited-memory BFGS — best for small N, high precision |
119139
140+ ### The ln(N) Bound and Optimizer Fingerprinting
141+ For a kernel-density entropy of the form $H = -\sum p_i \log p_i$ over $N$
142+ points, the maximum possible value is $\log N$, attained when the induced
143+ distribution $p$ is uniform ($p_i = 1/N$ for all $i$). Equivalently, when every
144+ point has identical local kernel density $\rho_i$, the entropy saturates its
145+ information-theoretic ceiling:
146+
147+ ```
148+ H_max = ln(N)
149+ ```
150+
151+ This is just the standard Shannon bound applied to the per-point density
152+ distribution, and it grows ** proportionally to $\ln N$** as the point count
153+ increases. Empirically, runs of the maximizer asymptote at this value for every
154+ geometry we have tried — sphere, torus, cube, saddle, or arbitrary STL.
155+
156+ #### A Sparse Fitness Landscape
157+
158+ What makes this problem interesting (and a little strange) is that the
159+ condition for achieving $H = \ln N$ is ** highly underdetermined** . The
160+ objective only requires that all $\rho_i$ be equal; it says nothing about
161+ * where* the points sit, only that each one must see the same total kernel mass
162+ from its neighbours. On a curved or non-trivial manifold there are typically a
163+ continuous family — sometimes a high-dimensional manifold — of configurations
164+ satisfying this constraint.
165+
166+ In optimization terms, the fitness function is ** sparse** : a vast set of
167+ distinct geometric arrangements all sit at the same global optimum. The loss
168+ surface has a large, flat, connected (or near-connected) optimal set rather
169+ than a single isolated minimum.
170+
171+ #### Optimizer Fingerprinting
172+
173+ Because the optimum is degenerate, the * path* an optimizer takes through
174+ configuration space determines * which* extremal configuration it lands on.
175+ Running the same problem with ** Adam** , ** QQN** , and ** L-BFGS** — or even the
176+ same optimizer at different learning rates or temperatures $\tau$ — reliably
177+ produces visibly different point arrangements, each of which achieves the same
178+ $H \approx \ln N$ value.
179+
180+ - ** Adam** tends to produce slightly noisy, isotropic, lattice-like packings.
181+ - ** QQN** carves out smoother, more symmetric arrangements with visible
182+ curvature-aligned structure.
183+ - ** L-BFGS** snaps quickly into crystalline, near-perfectly-regular
184+ configurations, often with sharp symmetry groups.
185+
186+ The resulting geometry is, in effect, a ** fingerprint** of the optimizer's
187+ internal dynamics — its preconditioner, momentum, and step-selection rules
188+ projected onto the manifold of entropy-maximal configurations. This is a
189+ previously unremarked-upon (and admittedly ** useless** ) property: the entropy
190+ value tells you nothing about which optimizer produced it, but the * shape* of
191+ the resulting point cloud does. You can identify the optimizer by looking at
192+ the picture.
193+
194+ This is a small, charming consequence of optimizing a sparse objective on a
195+ continuous manifold, and the lab is a convenient place to play with it.
196+
120197---
121198
122199## Visualisation
0 commit comments