statistically significant?

[mrjones]

hi. it has been too long since i have done stats. could anyone verify whether the following is a reasonable test for statistical significants?

i have a system which has equidistant SL and TP level. for simplicity, spread is not taken into account. for random entry, i expect the win rate would be around 50%.

i test the system. after 2000 trades, i see a win rate of 58%.
i calculate the z-score. (0.58-0.50)/sqrt(0.5(1-0.5)/2000) = 7.2
so 7 standard deviations from the mean.

can i then conclude that my system has found non-random market behaviour? and that it's ridiculously unlikely that such a result is possible by chance alone?

 

[Mr Doe]

It's ridiculous that you are taking this numbers (z score etc.) seriously. They can at best serve as a benchmark, nothing more, nothing else.

 

[joel_b]

That makes no sense at all. Your mean is made up, you can't assume a win rate. that's not a standard deviation calculation either. Stats should use actual data. You should be looking at benchmark returns vs strategy returns...like an equal weighted basket of the securities.

Win rate is good to know...but you can win 58% of the time and still lose money...it doesn't tell you if you're profitable, which is kind of the point, isn't it?

 

[mrjones]

That makes no sense at all. Your mean is made up, you can't assume a win rate. that's not a standard deviation calculation either. Stats should use actual data. You should be looking at benchmark returns vs strategy returns...like an equal weighted basket of the securities.

Win rate is good to know...but you can win 58% of the time and still lose money...it doesn't tell you if you're profitable, which is kind of the point, isn't it?

hi. thx for your response. it's in the context of hypothesis testing.
null hypothesis: system is no better than random entry.
conditions: stoploss and takeprofit levels are equidistant from entry. win yields x points. loss yields y points. where these values are fixed and x=y.

assuming the null hypothesis were true, it's theoretically no different to a cointoss. so z = (p - p0)/sqrt(p0(1-p0)/n)
where p0 is the mean win rate if the null hypothesis were true, p is the actual mean win rate, and n the number of trials.

is this wrong? basically, all i'm trying to prove is that it's highly likely that my system ain't a coinflip for a high-frequency trading system (which goes both long and short). it's not a long-term investment thing, it has absolutely nothing to do with benchmark yields.

 

[iHateVariance]

hey 99% confidence interval(for win rate if ur trades r iid with 2000 trades) is aprx [55%,61%]. u can check distribution of runs vs iid. at higher freq i think it makes sense to increase ur sample size to 10000+ to get better bounds and check convergence.

 

[joel_b]

ahh I didn't read that right. My point was just regarding practicality, trading costs etc. Win rate isn't the only measure to use, that's all. A Benchmark does not have to be based on a long-term metric, you can define it however you want.