self-correctionRound 12 · Step 42026-06-06
Bradley_Tm deflation: 0.826 → 0.655
The published melting-point r was partially inflated by a v1 hydrogen-bond bug: a 1.2σ donor threshold miscounted aromatic π-clouds as false donors (~45% false positives on aromatics). The Bradley cohort is aromatic-rich, so that spurious signal leaked into the fit. After a v2 σ-degree atom-role fix, the honest r is 0.655. The other 7 production kernels don't select donor/acceptor and are unaffected.
ruleFind your own bugs and report them openly — that self-correction is the credibility moat.
method lessonRound 11–122026-06-06
2-term overfitting on scaffold-blind (ESOL, CompSol)
On the 25% blind held-out, the ESOL and CompSol 2-term gates failed — r_cv collapsed from ~0.81 train to ~0.02. Diagnostic v2 confirmed the systems are well-conditioned (cond#=1.3), so this is GENUINE statistical overfitting, not numerical instability. The published 2-term r is a full-data upper bound; the 1-term fallback is the honest lower bound.
ruleThe 5-fold gate (train−cv ≤ 0.10) correctly catches overfitting. Don't loosen it to make a number look better.
method lessonMNSol_octanol2026-06-06
Form-stable but magnitude-noisy
An initial 'overfit suspect' verdict was revised after a 30-seed bootstrap: the cavity+g_polar form is selected in 89% of resamples (H-bond features never), but the magnitude is noisy because the blind cohort is tiny (n≈8). That's sampling variance, not classic overfitting.
ruleDistinguish form stability (bootstrap across resamples) from generalization (k-fold CV). Report both; widen the CI rather than retire a stable form.
method lessonRound 82026-05-31
Calibration diagnostic preserves the honest taxonomy
Hypothesis: maybe the low-lift kernels are just calibration-compressed. We fit y = α + β·ŷ on grouped-LOO predictions and found slope ≈ 1.0 everywhere — no compression to recover. The 28–30% MAE-vs-NULL floor is real signal. The B.UNDERFIT category was not bent.
ruleNever manipulate verdict categories to rescue a kernel; audit the predictions first.
negative resultRound 92026-05-31
Tier-1 shape features: negative
Three orthogonal shape descriptors (globularity, rotatable-bond count, halogen count) were tested under the full gate. Δr = 0.000 on all three kernels; none entered a winning pair; the negative control stayed flat.
ruleFeature additions face the same rigor as kernel claims. 'r went up' is not success.
negative resultRound 10 · Program 02026-06-01
Explicit polarizability α: real but sub-bar
Isotropic polarizability (GFN2-CPSCF via dxtb) was added as a 10th feature for the dispersion kernels. It is genuinely complementary to donor (a {donor+α} 2-term reaches r=0.586, CI excludes 0, 100% fold-stable, negative control flat) — but it fails the +30% MAE gate on every testable kernel, so it was NOT productionised.
rulePre-classify which kernels a feature should help before running it broadly; negative controls are mandatory.
method lessonRound 62026-05-29
Nested CV exposes winner's curse
Llinás_logP looked like r=0.81 under the old anchor-LOO protocol. Per-fold pair selection under nested CV collapsed it to r=0.57. Real 2-term lift survived on only 3 kernels (ESOL, CompSol, MNSol_octanol); the rest hit a 1-term ceiling.
rulePer-fold pair selection is mandatory before any multi-term claim.