My bot has logged over 10,500 trades across 157 strategies. That sounds like a lot of evidence. It isn't, and treating it as if it were has misled me more than once.
These rules live in a notes file that my coding assistant reads before every analysis. None of them is a stylistic preference. Each exists because ignoring it once produced a wrong answer.
1. Measure in points, never rupees
My position size has changed several times — once by 27x in a single weekend and back again. Any rupee figure — total P&L, drawdown, average win — mostly measures how big the position happened to be that month.
So everything is measured in premium points per unit traded. It is the only measure where June and September are comparable.
2. Resample days, not trades
Trades on the same day share the same regime, the same opening range and the same news. They are not independent. A t-test or bootstrap over trades will report significance that isn't there.
I resample whole days, and every result states the number of independent days next to the number of trades. Thirty trades across three days is a three-sample result.
3. Expect false patterns, and gate them
Seven strategies × twenty filter fields × three market regimes is about 400 comparisons. At the usual 5% threshold, that is roughly 20 "findings" per pass that are pure noise.
A real example: one breakout strategy showed a strong average in weeks labelled downtrend. Inside that bucket the daily results were +12.75, +10.31, −5.06, −4.69 and +0.81 — one good Monday and Tuesday, in a single week, inside a strategy that lost in every other cut. Always look at the per-day values before believing a bucket.
4. A findings register with an out-of-sample gate
Any pattern found by scanning goes into a findings register as a hypothesis, written as a precise, machine-checkable rule. It is then re-tested only on trades that closed after it was written down — data it was not discovered on. Only if it survives does it become a proposal.
The register evaluates the rules deterministically, so the goalposts can't drift from one day's analysis to the next. The rule format is the same one the live gate uses, so a finding that survives can become a live rule with no translation.
5. A real edge is a plateau, not a sawtooth
When sweeping a parameter, look at the whole curve, not the best cell. One exit rule looked strong at its best setting. The sweep read +6.56, +3.78, −1.45 and +6.91 across neighbouring values. When adjacent settings swing from best to worst, the parameter is fitting noise.
6. Check whether the result is carried by a few trades
Before proposing a profit cap on one strategy, I checked its distribution. Its top 10% of trades contributed 176% of its total points — the rest, together, lost. Capping profits would have removed exactly the trades carrying it, and every cap I tested made it worse. Cut losers; don't cap winners unless the distribution says so.
7. Report the null result, and don't read n = 1
"Nothing cleared the bar this week" is a correct, useful outcome. Manufactured findings are worse than none.
Likewise, a single outcome is not evidence either way. An AI veto in my allocator once blocked a trade that would have lost ₹10,179, and also blocked a later one that would have made ₹12,460. The honest summary is "sound reasoning, outcome mildly against it" — not a verdict on the veto.
8. Don't automate what is already working
I assumed my automated exits were better than my manual ones. When I split exits by who closed them for one strategy family, manual exits from the dashboard captured a median 69.4% of the peak available profit. Automated ones captured −58.4%. Every rupee of that family's profit came from my own discretion.
The right change was an alert when a position starts to roll over — not a replacement for the judgement.
9. Verify against the data, not against what the code should do
Claims about what the bot did belong to the trade log and the logs, not to reasoning about what the code is supposed to do. Three trades in my log show a profit even though the exit price is below the entry price — because a partial exit booked profit earlier at a different price. Anyone computing results from entry and exit prices alone would get those three wrong.
Owning the corrections
The final rule in the file is simply: own mistakes plainly. Corrections have reversed my conclusions more than once — a strategy ranking, a portfolio comparison, and two bugs found while reviewing a fix. Every one of those reversals came from applying the rules above to my own earlier work.
If you only adopt one of these, make it the findings register. It is the difference between research and a very sophisticated way of fooling yourself.