database question

[melioris]

I have been thinking about this for a while and I want to get feedback before I do it. I am thinking of deleting my tracker database and starting over. Ok, maybe not deleting, but not using any hh gathered before August 1, 2008. I got this huge database, and I figure that 99% of it is worthless, or worse, misleading. A lot of the regs I play against have adjusted their game over the past 6 months and the number of hands I have against them is so large that those adjustments are hidden by an avalanche of now meaningless HHs.

Thoughts?

[de-coder]

I have a few thoughts on this, but I'm heading out the door for work - I'll get back to this later today (or earlier if I can grab a few minutes).

[hackers238]

I read a discussion a little while ago about this, and the consensus seemed to be keeping only the 2000 latest hands. I'll try to dig it up...

[Talking Poker]

2000? That seems silly. Or do you mean only looking at the last 2000 hands from EACH of your opponents? That, I'd buy.

[de-coder]

OK, I'm back...

I'm assuming we're talking about PT - it's the only manager I have experience with. I'm also assuming you're using postgres for the database engine - if you're not... well, I've ranted on that a few times here already (do a search if needed).

Before you go ahead and create a new database you might consider using the built-in tools to purge the older HH files:

Before you begin - first back it up, then run a vacuum full/analyze

Under Utilities -> Database Maintenance and Options click Advanced Purge. From there you can get pretty specific regarding hands you want to get rid of - in your case a date range is what you're looking for. I've not run this myself, but I would expect it to take a while.

I highly recommend running a separate database for observed hands - you should purge this one as well.

You also have the option of setting a filter on both your PT screens as well as (more importantly) PokerAce HUD. For the HUD it's under: options -> filters - then click the Show Hands From The Last [0 ] hands Only checkbox and put a number in the box (like ~2500 I would guess). That should give you the same effect as the purge.

If you do continue on and create a new database I hope you save your HH files so can import the ones you want. If not, AND you've chosen to store the entire HH file in the database, you can extract them from the database. I'm not sure right now how you'd parse them for date though - there might be a way, but I'm not sure.

For maximum performance of the database - here are a couple of things that I do:

- I run Postgres on a different server - mine is a linux box. This does a couple of things for me - it offloads the SQL processing to another machine and postrgres runs faster/better on linux.
- I keep separate databases for each site and the observed hands for each site. They say you don't have to, but I do - and I believe I get better performance from it. It's a bit more work, but I think it's worth it.
- I run maintenance on my databases very regularly. This keeps the databases clean and tidy and fast - particularly when datamining. Inserting new records into the database is more expensive than reading as the server needs to maintain the indexes - running the maintenance makes this much faster.

[de-coder]

Must be - that's the way the filter works in PAhud.

[melioris]

excellent post decoder. I don't know shit about database stuff. While I am appreciative of your post, I was asking more along the lines that sometimes I have a shitload of hands on a player, but i think the player has totally adjusted his game and now the stats are irrelevant yet, since my total hands on the player over the last two years or so is so large, I will never see the adjustment reflected in a change in the stats.

[de-coder]

It looks like all the purge options are pretty global. You do have the option to selectively purge certain sessions, but it may also purge some of your own details if you were at the same table (not a problem for observed hands though). You might try that. I haven't seen a way of removing a certain number of hands for a certain player.

OK, so if you go to the Database Maintenance And Options you can select a player from the database. It will then display all the sessions that player has played (it's sorted by start time, but you can click on a header and change it). Select the sessions you want to delete (in your case I guess the older ones) and click Purge Session(s). If you only do your observed database you don't need to worry about your own stats (including things like lifetime win/loss and stuff).

That should get you to where you want to be.

[hackers238]

Yeah, 2000 latest from each opponent. I must have found this in another random forum while browsing because I couldn't find it again.

[Talking Poker]

If this needs to be done manually for thousands of players, I'd rather start from scratch.

Use the freaking PAHud filter. 10 seconds beats the hell out of 100 hours.