06
The lab that said no, eight times
The closest hypothesis was genuinely profitable, plus 10 to plus 19 basis points per episode across 492 out of sample episodes, and the lab still killed it. Profit is not the same thing as proof.
- Python
- pandas / numpy
- pytest (86 tests)
- 1 minute FX data
- 4.45 GB, licensed
- deflated Sharpe
I built a backtesting laboratory whose entire product is answers I can trust, then pointed it at eight of my own trading ideas across currencies, crypto, metals and equity indices. Every hypothesis was pre-registered before it touched data: a falsifiable claim, a statement of who is on the other side of the trade losing money, explicit kill criteria, and a fixed budget of attempts. Every run lands in an append only ledger, so the statistics stay honest about how many things were tried.
Two tripwires keep the instrument itself honest. A strategy with no edge must earn nothing, and a strategy that cheats by peeking one bar into the future must be caught loudly. I calibrated in both directions with planted synthetic edges: it never cried wolf across 140 checks, and it reliably finds a real edge once that edge is worth roughly twice what trading costs.
The verdict after eight hypotheses was eight kills, no false positives, and a written reason for every death. That is the skill I am actually selling: knowing when the answer is no, and being able to prove it.
Gate A: a zero-edge coin on zero-drift bars must lose net almost exactly what it pays in costs, ~zero gross. Gate B: a deliberate lookahead signal must produce an absurd Sharpe (leaks are VISIBLE), and shifting it +1 bar must destroy it. Hard rule 1: never modify tests to make them pass. Hard rule 5: the trial ledger is append-only. Every backtest counts against the deflated-Sharpe bar, forever.
The kill board
Eight pre-registered ideas, eight verdicts. A lab that never kills anything is not a lab, it is a sales pitch.
H4 is the highlighted one, and the hardest to let go of. It was profitable, plus 10 to plus 19 basis points per episode over 492 out of sample episodes, and it still fell short of the bar once every earlier attempt was counted against it.
Statistics and charts are our own. The underlying 1 minute price data is a licensed purchase and is never redistributed here.
How it was done
Pre register before touching data
Each hypothesis is written down first: a falsifiable claim, a statement of who is on the other side of the trade losing money, explicit kill criteria, and a fixed budget of attempts. You cannot quietly move the goalposts after seeing the result if the goalposts are already published.
Count every attempt, forever
Runs land in an append only ledger. Every backtest ever run counts against the significance bar, which is what stops the twentieth idea looking impressive purely because nineteen were tried before it.
Test the instrument, not just the strategy
Two standing tripwires: a strategy with no edge must earn nothing beyond its costs, and a strategy that deliberately peeks one bar into the future must produce an absurd result that is impossible to miss. If the second one ever goes quiet, the harness is leaking.
Calibrate in both directions
Synthetic edges of known size were planted to check the lab finds real signal, and pure noise was run to check it does not hallucinate any. It never cried wolf across 140 checks, and it reliably detects an edge once that edge is worth roughly twice what trading costs.
What this does not tell you
Every dataset has an edge. Here is where this one stops, said plainly, because a number you have to qualify is worth more than one you do not.
- Eight kills is not proof that no edge exists in these markets. It is proof that these eight specific ideas did not clear a deliberately hard bar.
- The hardest case to let go of was genuinely profitable in out of sample testing and was still killed. That is the system working as designed, but it is worth being clear that a lower bar would have shipped it.
- The underlying minute by minute price data is a licensed purchase. The statistics and charts here are mine to publish; the data itself is never redistributed.