Poker Math Part 2: Sample sizes
 

Found this short article. Interested to hear what the mathimatically inclined (especially the non-poker academic type) think of this. In terms of self analysis, this has some merit.

the link...



the article...

Sample size revisited
May 10th, 2008

Two plus two has another thread on that perennial question: How many hands do I need to play to know whether or not I’m a winning player?

How many hands to know…

----------------------------------------------


if you’re a winning or losing player? 5,000? 10,000? ie. how long does it take for the bad beats/luck to even out enough so that you have an accurate idea of how you’re doing at poker?
The responses are the standard ones. Confidence intervals. Poker has large standard deviations. You need large samples. Blah, blah, blah.

None of the responses address the fundamental problem with trying to use classical confidence intervals to determine whether your win rate is “significantly” bigger than zero — non-stationarity. Your results stream doesn’t come from a constant distribution, the underlying mean and variance of the process changes.

The bigger the sample you use the more likely you’re dealing with a mixture of distributions that is extreme.

Bigger samples don’t give you more reliable estimates of your win rate. Classical confidence interval analysis requires an assumption of a sample stream of independent, identically distributed observations.

The answer is to look at smaller samples, not larger samples.

Pick one hand a day. Not randomly, but pick the one that involved the most money, the biggest one hand swing you had that day. Won or lost doesn’t matter. Then look at that hand in great detail. Did you make any mistakes? Should you have made that pot bigger? Should you have put less money in that pot? What prior information did you have about the opponent hand distribution? Was the result consistent with that? etc, etc.

That kind of analysis, a hand a day for 30 days, will give you tons more information about whether or not you’re a winning player than confidence intervals derived from the results of a million hands.

And it’s a statistically sound approach while confidence intervals are not statistically sound when you are not sampling from a constant distribution.