need help with poker betting AI

sneakyimp

hi all:

I've been writing a poker game (texas holdem!) and I want to make some worthy computer opponents (with different behaviors).

I need to write some AI! Never done that before.

I basically want to write a function

function robo_bet() {
}

that will evaluate a player's 2 pocket cards and check/call/raise/fold accordingly.

If you don't know anything about texas holdem, it's a great poker game. more info here: http://www.texasholdem-poker.com

the way it works is like this:
1) the dealer button is moved clockwise so the next player gains the advantage of dealer: last bet on all rounds after the first.
2) people ante up / some forced betting occurs
3) 2 cards are dealt to each player
4) A ROUND OF BETTING...this is where my function gets called the first time
5) 3 cards are turned face up on the table ("the flop"). these community cards, combined with the pocket cards form a poker hand. the fact that many cards are shared makes for interesting betting.
6) BETTING ROUND 2 - my function called again
7) 1 more card flipped over on the table ("the turn"). there are now 4 community cards
8) BETTING ROUND 3 - my function
9) 1 last card flipped over ("the river") for a total of 5 community cards
10) BETTING ROUND 4 - my function called one last time
11) hands are evaluated and the pot gets distributed to winner(s).

ideally, the function would respond differently depending on a variety of factors

the round of betting...you have a different amount of cards at each stage (1st round 2 cards, 2nd round 5 cards, 3rd round 6 cards, 4th round 7cards). If you have a terrible hand with 7 cards you better not bet!
the amount needed to stay in the game. If the current wager to match is large, this will discourage people with poor hands. but if the POT is enormous, a larger bet might make sense. if you have an incredible hand, large bets make complete sense...but don't bet too large or you'll scare away the other players! you want to rope them in as long as possible.
they player's rank in the betting order. if you bet first, you don't have much information about other people's cards. if you bet last, you've seen everyone's response to the current card situation.

basically, I'm kind of at a loss as to how I should proceed. I would imagine that calculating some indices based on available data would be smart. potential inputs are:

pocket cards (array of 2 cards each with value and suit)
table cards (array of 0 to 5 cards each with value and suit)
which round
the current bet amount that you must call/raise/fold
the size of the pot
the number of other players
how much money you have

the output:
the amount of your bet:
0 (check / fold)

0 (call or raise)

thoughts?

sneakyimp

Here's a much more elaborate rule description.

http://www.partypoker.com/games/how_to_play/texas_holdem.html

their poker game is quite nice.

drawmack

What you want isn't AI it's probability and statistics. You'll probably want to write two functions one for use before the flop is set and one after.

Then all you need to do is calculate the probablity that the other players at the table have a better hand then you do. This isn't hard for the first function as you're only taking two cards into account:

cards are ranked 1 - 13 (Duece - Ace).

Now the chances of someone having a better card then you do are (13 - n)/13, where n is the number of your card, for each card in your hand. So you take both of these, add them together and divide by 2 to get the probablity that someone has a better hand then you do.

For example if you had a 5 and a King. Your hand would be worth ((13-5)/13 + (13-12)/13)/2 = (8/13 + 1/13)/2 = (9/13)/2 = (9/13)*(1/2) = 9/26. Which is about 35%. So there is a 35% chance that someone has better cards then you do at this point.

For the second one you'll first have to figure out which hand you have (i.e. flush, straight flush, etc.). Then you'll know (from a little research) what the probablitity is that someone can beat your hand. Just return this probability.

The neat thing about donig it this way is that each player can have their own percentage of when they bet/meet/fold/etc. So with two fairly simple functions you'll create the impression of advanced AI.

NOTE: You'll probably want to create more then two functions as the second function will probably use helper functions that test for each type of hand.

sneakyimp

Thanks SO much for your suggestion. I will see if I can implement that today. My suggestion that this might require AI was more based on the gaming community concept of "opponent AI". It might not pass any Turing test but it does have some notion of intelligence to it.

I bet the algorithm you suggested will be fairly good. I think the comparison is a bit simple though. Consider a hand of A / 2 versus 6 / 9. Even though your algorithm would evaluate them about the same, there are some differences:

My reasoning goes like this:
1) if you end up with a nothing hand, ace hand wins ever time. Ace hand is better.

2) if both hands ended up with a pair by matching either pocket card, there is a 50/50 chance that either hand will win. If a pair appears on the table, ace hand will win with ace as a kicker. Ace hand is better

3) 3 of a kind logic is similar Ace as kicker makes ace hand slightly better.

4) two pair? match pocket cards, ace hand wins. match one of the pocket pairs? 50/50 (this is getting a bit beyond my ability to think properly about probabilities).

5) straight? depends where the straight happens. seems to me 6 / 9 hand is stronger. What about measuring the "straight potential" of a 2-card hand? How might that be done?

6) flush? ace kicker might make ace hand better. in the case of off-suit pocket pair (2 original cards are different suits), chances are split 50/50 between the two hands.

and so on.

Anyways, my intuition (and Celebrity Poker Showdown) tell me that a good pocket pair is 2 high cards--ideally two sequential ones (i.e, 6 / 7 or Q/K) and of the same suit. I would like to construct something that examines the "outs". an "out" is a card that will improve your hand. if you have Q/K of Diamonds, outs might be jack, ace, q, k, or another diamond card.

I've googled around...apparently the university of alberta has a seriously good poker algorithm going (it's like their version of Deep Blue) and some other postings i've seen talk about calculating the likelihood of outs and stuff.

i just don't know how to string all the logic together. I'll go googling some more.

THANKS!

drawmack

You're over analysing. When all you've got is your two cards all that you can really go on is the probability that someone else at the table has better cards then those two.

You really can't get too detailed without running into some very complex programming. For example:

I have 6h, 9h and someone else has As, Ad.

Which would you say is better?

But then what if the flop turns out to be: 7h, 8h, 10h.

Now which one is better?

To be completely accurate you'd have to figure what hands are possible with the two initial cards dealt, that's not as easy as it sounds. Then you'd have to calculate the probability of each of those hands coming up, and don't forget to take into account that you'll have n*2 unkown used cards where n is the number of other players.

So you see really simplicity is the best bet when you're doing the initial round of betting.

sneakyimp

I found a very interesting site. This guy ran a simulation of 5 billion 10 player poker games (and 2 player and 8 player) and reported a table of how likely each hand was to win.

http://wizardofodds.com/games/holdem.html

Given these results (which take into account the suit as well as card value), it seems to me that it might be pretty easy to construct a card hand lookup table that gives you a pretty accurate readout of your chances of winning.

It is worth noting that this approach is VERY similar to what drawmack was suggesting...only with some realistic probabilities that take into account the suits of the cards.

Based on that probability and the size of the pot, you can make a pretty intelligent assessment of the expected return. My probability is a bit rusty, but i seem to recall that (CHANCES OF WINNING) * (AMOUNT OF JACKPOT) = (EXPECTED RETURN). If your expected return is less than the cost of staying in the game, it's time to fold.

Once you get to the flop it seems worthwhile to me to not only assess whether your hand has anything going for it but also how many "outs" you have...then you can use this table:

http://www.texasholdem-poker.com/odds_chart.php?chance_format=to_win_ratio&decimals=3&Display=Display

drawmack

Yes the probability in the above post is correct. However what you can do is take that equation and have +/- values for your different automated players.

Maybe Saftey McMasterson stops betting when Expected Winnings < Sum of all bets + (Current Bet * .1)

While Smart Sam stops betting when Expected Winnings < Sum of all bets

Then Silly Sal stops betting when Expected Winnings < Sum of all bets - (Current Bet * .1)

sneakyimp

actually, your expected winnings will always be less than the jackpot unless you have a 100% chance of winning.

imagine that you are rolling a die (six-sided 😉 ) and the jackpot is $6. your expected winnings amount to $1 (1/6 * 6 = 1). if you have to pay anything more than $1 to get your chance, it's not worth it. so the betting logic might run more like this:

Safety bets when the bet amount is < (expected winnings .9)
Smarty bets when the bet amount is < (expected winnings)
Silly bets when the bet amount is < (expected winnings 1.1)

I also hope to make the logic less linear than that...introduce some random element to it so that the computer players aren't so predictable.

drawmack

Possibly I should have been more clear:

Sum of all bets = Sum of my bets

sneakyimp

once you have wagered money, it no longer belongs to you and should be considered part of the jackpot, so previous bets bear on your current decision only inasmuch as they have contributed to the jackpot total--at least that's how the theory goes here:

http://www.texasholdem-poker.com/odds.php

i'm still confused by your formulas...perhaps you could explain them to me a little more?

another interesting observation: in a 10-player holdem game, those stats from the wizard of Odds say that even a pair of Aces only has a .3138 probability of winning. if we use our simple logic previous described here, my computer betting algorithm would never bet on the first round. check it out:

ROUND OF POKER!
small blind antes $1
big blind antes $2
3rd player runs my betting algorithm:
  jackpot:  $3
  pocket cards:  A spades, A hearts which has probability .3138 of winning
  expected return:  $3 * .3138 = $0.94

since the minimum bet is $2, my computer algorithm would never bet. i have this funny feeling that the math is telling us something profound here...namely that poker doesn't pay! heheh.

but really...if you consider that players after you might bet, the equation changes a little, but i'm not quite sure how to factor this in.

rather than over-thinking this, i think i will just use the probability tables from the wizard of odds and make some kind of probability spread from the best hand to the worst. the algorithm will generate a random number and compare it to some function i concoct. a pair of aces will ALWAYS result in a bet...conversely, a 2/7 unsuited will always result in a fold or check. in between values will have an intermediate probability of play or fold depending on how they compare to the average probability of .123253.

J/5 suited, for instance, has win probability of .123 so there would be a 50/50 chance of folding on that one.
A/K suited is much higher than average and would therefore have perhaps near 100% chance.

how does that sound? any input would be appreciated.

drawmack

Here is what I was thinking.

First some variables for notation:

p = pot
b = bet
sb = sum of monies I have placed into the pot
o = odds of winning hand
ew = expected winnings
cb = amount of current bet
os = offset

So now as the hand progresses each player keeps track of the money they have bet.

Then when it is time to make a betting decision they look at it like this

o * p = ew

if (ew < (sb + cb + (cb*os)))
    bet
else
    fold

The only place in this algorithim for player differences is in the offset. Offset can be positive or negative. 0 is the most logical choice. A negative number represents a risky player who will keep going long after they are expecting (statistically speaking) to loose money on a hand. A positive number represent a safe player who stops betting long before they expect (statistically speaking) to loose money on a hand.

So yes money that has been placed into the pot is no longer the player's money, but it should still be used in determining future bets. Because if it is not then on a small stakes game the computer will never stop betting even if they have only a somewhat descent hand because the pot grows much more quickly then the bet, thereby making the expected winings always greater then the current bet.

sneakyimp

coolio...i will walk thru a scenario here.

10 players are playing. it's a $2/$4 game so small blind is $1, big blind is $2. for first player to act (clockwise from big blind), pot is $3, required bet is $2. this player's hand is pair of aces--the strongest hand--with probability of .3138

p = $3
sb = 0
o = .3138
ew = 94 cents
cb = $2
os = 0

// your algorithm...i think you meat to have a > sign

if (0.94 > 2)
    bet
else
    fold

this would basically mean that all computer players would always fold--even if only 2 players were playing and your chances of winning with pair of aces were the much higher value of 85%.

let us assume that the pot is quite large somehow...a series of raises have jacked up the pot to $30 and i'm asked to bet for the 2nd time:

p = 30
sb = 2
o = .3138
ew = 9.41 // equal to 30 * .3138
cb = 6
os = 0

// your algorithm
if (9.41 > (2 + 6 + 0))
    bet
else
    fold

i see what you mean about endlessly betting as the pot grows large...we need to make two separate decisions i think. firstly, to call or fold. secondly, if calling, what about a raise?

I did some graphs in excel which i have attached that plots EW and SB+CB for some different scenarios:
1) no one folds
2) one person folds each raise
3) 2 people fold each raise

the results suggest that we need a different kind of function...it does show promise...your investment grows linearly while the pot's growth shrinks as people fold--but it needs some more thought.

drawmack

Yeah, it was pretty quick and not deeply thought out. But it got you going down the right road.

sneakyimp

i'm still having all kinds of trouble generating proper functions to halt betting.

having done a bunch of spreadsheet work, i'm starting to think that something more similar to your original suggestion will be smartest....preflop, just evaluate your 2 cards and ignore everyone else's betting.

as for preventing rounds and rounds of stupid betting, i'm tempted to just implement a limit on the number of raises per round. this is quite common in limit poker (typically 4 raises per round max) and just leave it at that. my intuition tells me that this approach might be more bluff-proof than worrying about what the bet currently is.

i would really like to avoid predictable computer players so i'm considering introducing some random element into the decision making....general algorithm preflop might proceed as follows:

myConfidence = probThatMyHandWillWin;
myFear = some value between 0.4 and 0.6

if myConfidence < myFear then call
if myConfidence - myFear > 0.2 then raise

how does that sound?

i will handle post flop betting after i deal with this.

sneakyimp

oh...one more thing:

the probabilities from the wizard of odds don't have pair of aces as 100% winner--for good reason. i've often lost to a flush with precisely that hand.

at any rate, to fold on preflop with a pair of aces should NEVER happen. i have taken to scaling the probabilities to range from 0% to 100% like this

scaledProb = ((rawProb - lowestProb)*100)/(highestProb-lowestProb)

just wondering if that sounds legit or like math foolishness.

drawmack

First, I think the randomness is a good thing, but you should also be able to vary the varience (i.e. a safe better might use 0.475 < x, where as a normal player might use 0.45 < x and a risky player might use 0.4 to decide call. On the raise the same variance ability should be built in.

Originally posted by sneakyimp
scaledProb = ((rawProb - lowestProb)*100)/(highestProb-lowestProb)

just wondering if that sounds legit or like math foolishness.

First off you scale probability by multiplying it by 100, period.

I'm not sure what you're doing with the math cause I don't know what rawProb, lowProb and highProb are?

sneakyimp

the wizard of odds stats say that in a 2-player game, pair of aces has 85% chance of winning. 2/3 unsuited has 29% chance. it seemed ridiculous to me that my computer player would ever fold a pair of aces or play a 2/3 unsuited (although i know people play this hand occasionally just for kicks).

SO, i came up with a formula which will scale those wizard of odds probabilities so that AA is 100% and 2/3 unsuited is 0%

scaledProb = the new value, scaled
rawProb = the original wizard of odds probability
lowestProb = the lowest odds...2/3 unsuited is .29
highestProb = the best odds, pair of aces is .85

for aces, this gives
(.85-.29)*100/(.85-.29) = 100%

for 2/3 unsuited, this gives
(.29-.29)*100/(.85-.29) = 0%

for a middling hand of 4/K unsuited, this gives
(.50-.29)*100/(.85-.29) = 37.5 %

legit or madness?

I do plan to give computer players difference characteristics, just like you described before.

Shrike

Nice going Drawmack, another maths boffin in the house 🙂

I play Texas Holdem alot with work mates, don't win much though. This 2 of spades, 7 of clubs is a good, right? 😃

sneakyimp

i hates the maths. it's gotta be done though.

at any rate, i have a working preflop betting algorithm...it's pretty simple. it takes a peek at the ranking of the hand is given by the wizard of odds and applies a small adjustment based on the pot odds. large pot with small be requirement will increase a bettor's confidence a little.

anyways, it seems to make pretty reasonable decisions preflop. so i'm on to post flop strategy. maths!

so, given two pocket cards and 3 or more table cards, we need to make a decision. i have a function which can assess pocket cards and table cards and return a hand value...0 (for simple high card) up to royal flush of 9. here are the theoretical probabilities of getting each of the various hands...don't recall where i found these:

royal flush: 0.00003232
straight flush: 0.00027851
4 of a kind: 0.00168067
full house: 0.02596102
flush: 0.03025494
straight: 0.04619382
three of a kind: 0.04829870
two pair: 0.23495536
pair: 0.43822546
nothing: 0.17411920

my plan for this algorithm is to evaluate a player's hand and determine if it already amounts to something. if so, a base confidence value ($myHandConfidence) will be calculated based on the hand. this raw confidence value will be the likelihood that one of the other remaining players (IF they make it to the river) will draw a higher hand.

for instance, if i have a pair after the flop, my raw hand confidence will be
1
minus the likelihood of someone getting a higher pair
minus the likelihood of someone getting a better hand than just a single pair.

if i have a pair of aces, that would be
1
minus 0
minus 0.38
approx: .62

if it was pair of 2's, then perhaps
1 minus .44
minus .38
approx: .17

i will then modify that raw amount further...adjust for hands that are no longer possible given the table cards, and also for the likelihood of landing outs, etc.

thoughts?