Estimator Correlation Study — First Real Result

The single highest-priority open experiment, per two independent assessments. First real run, 200 trials — found something immediately significant, and a real limitation of ablation studies along the way.

What was done

Ran the real, fixed StateEstimationEngine 200 times with varied random market data, logging every one of the 15 real estimators' raw scores per trial into a 200×15 matrix. Computed the real correlation matrix. Checked every pair, not just the ones that looked suspicious from variance alone.

What was found

Four estimators are perfectly correlated (r=1.0000) across all 200 real trials: channel_coherence, recovery_potential, channel_balance, schema_health. Checked the actual source, not just the statistic: channel_coherence, recovery_potential, and channel_balance compute the byte-identical formula (1.0 - std(S,D,I,C)/0.5) under three different names and docstrings. schema_health computes a weighted blend of that same coherence term and a linear transform of the same std (1 - std) — different code, but purely a function of the same single quantity, which is exactly why it correlates at r=1.0 despite not being literally identical code.

Two more estimators are near-perfectly correlated with that same cluster: weakness_detection (r=-0.98 with all four) and cross_channel_coupling (r=-0.98 to +0.96). Both are themselves built from formulas that depend heavily on the same channel-spread quantity.

Real implication: of 15 estimators advertised as independent diagnostic perspectives, at least 6 are effectively measuring one single underlying quantity — the standard deviation of the four S/D/I/C channel values — through different signs, weights, and labels. The true number of independent dimensions in this estimator suite is provably smaller than 15, likely closer to 9-10 once the remaining 9 estimators get the same check.

What this means for the architecture

This is exactly the kind of finding ablation studies are poorly suited to catch on their own — removing any single one of these 4-6 estimators would show "negligible effect" in an ablation study, correctly, but for the wrong reason: not because that estimator doesn't matter, but because five other estimators are silently carrying the identical information. An ablation study alone could have been misread as "none of these matter," when the real finding is "these are one estimator wearing six names."

What's still needed to complete this experiment

• Same correlation check across the remaining 9 estimators (particle_filter, ensemble, neural, symbolic, robust, tilt_interference, inter_channel_entropy, future_state_projector) — not yet done.
• Hierarchical clustering / community detection on the full 15×15 matrix, per the original proposal, to find the real natural groupings rather than checking pairs by eye.
• Compare the current NCFCA 4-group hierarchy (observation/structural/dynamics/composite) against what the data-driven clustering actually finds — the real test of whether the architecture's grouping reflects real structure or imposes it.

Reproduce

import numpy as np
from COMPONENT_A_TILT_SPECTRUM_FIXED import StateEstimationEngine

names = None
scores = []
for seed in range(200):
    np.random.seed(seed)
    e = StateEstimationEngine()
    md = {k: float(np.random.uniform(0,1)) for k in ['price','volume','fundamentals','liquidity','macro','sentiment']}
    e.fractal_brain.ingest_signal('normal market conditions', confidence=0.5)
    r = e.process(md)
    if names is None: names = list(r['estimator_outputs'].keys())
    scores.append([r['estimator_outputs'][n].get('score', 0.5) for n in names])

scores = np.array(scores)
corr = np.corrcoef(scores.T)
# Check channel_coherence vs recovery_potential -- should be exactly 1.0
i, j = names.index('channel_coherence'), names.index('recovery_potential')
assert abs(corr[i,j] - 1.0) < 1e-9
print("Confirmed: r =", corr[i,j])

This finding is what Fix 5 in the Component A Changelog was built to address — see that page for what was actually changed in response.