Research period: September 1, 2026
Historical anchor: #163
The previous experiment showed that changing the downstream computation could make the same spectral representation substantially more useful, but I had changed several things at once. The model handled magnitude explicitly, preserved phase through residual connections, and used phase-aware similarity in attention. Any one of those changes (or an interaction between them) could have been responsible for the improvement.
I separated those components through ablation: remove or replace one part of the architecture, rerun the comparison, and see which gains survive.
Sticky note — ablation: an experiment that removes or replaces one component while leaving the rest of the system as unchanged as possible. If the effect disappears, that component becomes a candidate explanation for the original result. Reference →
What survived
The phase-aware attention mechanism survived the ablation. Compared with the coordinate-only control, phase/Hermitian attention improved loss by about 0.3923 nat per original byte.
The other candidate explanations didn't survive in the same way. A phase-preserving residual connection by itself didn't clear the predefined statistical and materiality requirements, while explicit log-radius handling produced results that were bit-identical to its matched manual control.
This narrowed the result considerably. I no longer needed a broadly “polar-native” architecture to explain the improvement; the smallest surviving change was an otherwise mostly Cartesian processing block using normalized real-Hermitian similarity for the attention query/key comparison.
Sticky note — Hermitian similarity: complex vectors contain both magnitude and phase. A Hermitian inner product conjugates one of its inputs before comparing them, allowing their relative phase to contribute naturally to the result. Taking the real component and normalizing by vector magnitude produces the similarity score used here.
Illustrative example
These hand-chosen vectors demonstrate phase-aware similarity, not the ablation experiment.
Saved output is included so you can inspect the example without starting a kernel. Running it in Lab executes this example only.
import numpy as np
q = np.array([1+1j, 1-1j])
k = np.array([1+0j, 0+1j])
score = np.real(np.vdot(q, k)) / (np.linalg.norm(q)*np.linalg.norm(k))
print('normalized real-Hermitian similarity:', score)
normalized real-Hermitian similarity: 0.0
What this does not show
Hermitian and phase-aware attention already existed; this experiment doesn't claim otherwise. The result is narrower: for this spectral representation, replacing the ordinary attention similarity with a phase-aware comparison explained the improvement that survived the ablations.
It also doesn't establish a general rule that every representation benefits from specially matched operations. At this point I had one representation and one surviving operator pairing. What the ablation gave me was a much cleaner mechanism to carry forward: instead of changing an entire architecture in the next experiment, I could freeze the smallest component that explained the result and ask a new question without reopening this one.