Poker's Version of Big Blue?

[wade moore]

http://www.latimes.com/news/printedi...ostemailedlink

THE NATION
POKER-FACED
The opponent in the online card game might be a computer. 'Bots' are beatable because they miss human nuances, but they're learning.

By Joseph Menn, Times Staff Writer


Of the millions of gamblers who have rushed to play Texas Hold 'Em and other fast-growing poker games online, Roger Gabriel isn't the most intimidating.

The 30-year-old Newport Beach engineer started playing for money only a month ago. He lurks online at the tables for the chicken-hearted; even there, where the biggest ante is 4 cents, he can't win consistently.

ADVERTISEMENT

But Gabriel has a potentially powerful alter ego. In his spare time, he's perfecting a computer program to go online and play the game for him.

His BlackShark software is still a work in progress, but Gabriel has no doubt that such programs eventually will be championship quality. "In the future," he said, "robots are going to take over."

Gabriel is one of an increasing number of computer professionals who design poker robots, or "bots," that pose as human gamblers but can play endlessly without tiring or losing concentration — for real money.

Though not yet good enough to beat skilled humans consistently, these programs are seen as a threat by online casinos — all based outside the U.S. and out of the reach of American laws — and the gamblers who spend billions of dollars chasing big pots.

"There are already lots of robots playing online, and that's definitely unethical. They should identify themselves," said Paul Magriel, a veteran professional poker player.

The march of the machines will be celebrated in Las Vegas next month with the world's first money tournament for robots — and the $100,000 prize is drawing a handful of coders out of anonymity.

The emerging technology does more than raise the stakes for real people and online casinos. It also raises fundamental questions about how far computers have come in mimicking and improving on human behavior, and about how far they can go in the future.

Computer programs have conquered checkers, chess and, most recently, backgammon. By rapidly evaluating plays more moves ahead than a person can, computers routinely beat the strongest human players in those games.

This was demonstrated most dramatically in the classic 1997 match between world chess champion Garry Kasparov and Deep Blue, a 1.4-ton supercomputer built by IBM. The machine's victory marked Kasparov's first professional loss, and many took it as a depressing event for mankind. Even Gabriel, then studying artificial intelligence at UC Irvine, had been rooting for Kasparov.

Backgammon programs, which had to adapt to the random element of dice, grew so good by the late 1990s that they changed strategic wisdom built up over 2,000 years, influencing how the best humans play the game.

But poker — popularized recently by televised tournaments for pros and celebrity amateurs — is a far more human game, one in which psychology matters as much as probability.

That's why in poker there's no such thing as an absolutely correct play, except in retrospect. If someone, or something, bets heavily with a lousy hand and everyone else folds, that was the right bet.

This makes poker bot design fascinating to academics like Jonathan Schaeffer, a computing science professor at the University of Alberta in Edmonton who for 14 years has headed a project to build poker programs.

Schaeffer said cards were more likely than chess to produce computing approaches useful in the real world because poker players must deal with incomplete information. But before such research can contribute dramatically to solving real-world problems, Schaeffer said, it has to solve the challenge of poker — and that's several years away.

For now, only the poker players with the poorest skills — people like Gabriel, for instance — have much to fear.

Typically, a user signs on to an online game site manually, launches the poker bot and lets it run. Gabriel's BlackShark, for example, displays a window on his computer that collects information from the poker site and then calculates odds before making a bet.

Like most of his peers, Gabriel, who is working five nights a week to get BlackShark ready for the Las Vegas tournament, is an engineer first and a poker player second. He said his poor game skills are his biggest handicap.

"The hard part is: What if I've got two 10s? What am I going to do?" As he scans poker books for strategy tips, Gabriel is laboring to add an enormous set of rules telling the machine what to do with different cards and how to react to the frequency with which other players fold, call and raise.
Other robot designers, such as Ken Mages of Evanston, Ill., are further along. But though their electronic progeny may win at small-stakes tables, they usually fall apart when the human competition is stiffer.

After two weeks of programming, Mages said, "I could sit down at a 50-cent table, put 50 bucks in the account, go to bed and wake up with at least $75." The most Mages said he won that way was $250; he never lost.

ADVERTISEMENT

For two weeks this May, Mages sold his software for $60 a copy. After getting deluged with customer pleas for technical help — and a threat by one who gambled away $10,000 to send him the bill — Mages sold out to a business associate, Hong Kong engineer Ben Lo.

Mages then struck a deal with Los Angeles public relations executive Darren Shuster to set up the Las Vegas contest — dubbed the World Series of Poker Robots — and just after Memorial Day their partnership convinced Antigua-based GoldenPalace.com to put up the prize money.

Even though GoldenPalace bans robots, the publicity-craving virtual casino was a natural target, having spent $28,000 last fall for a cheese sandwich that was said to bear the image of the Virgin Mary. The sandwich is now on tour.

Organizers have further headline-grabbing gambits in mind: They plan to invite the winner of the human poker World Series to go up against the winner of their robot contest, though no one expects the computer code to triumph — at least, not this year.

Entrants in the robotic-poker tournament so far include Gabriel, Lo, programmers from Florida, Canada and Spain, and Hilton Givens of Lafayette, Ind., who started working on a robot more seriously after he was laid off from his software job.

Most of the confirmed competitors have run their programs on PartyPoker.com, which forbids such activity and confiscates the accounts of those it catches.

The cat-and-mouse game between robots and online game sites is not limited to poker. Whenever any free multi-user computer game gets big enough, cheaters use programs and other means to boost their rankings, collect useful game tools or exact revenge on competitors.

Gabriel, for one, cobbled together an unbeatable Scrabble robot, which he inflicted on Yahoo Games opponents. But the problem is especially acute for sites like PartyPoker, which has a million real-money players registered and so presents a tempting target.

And site parent PartyGaming might soon have to worry about spooked investors as well as spooked players. Gibraltar-based PartyGaming, which reported $350 million in profit last year, is gearing up for a multibillion-dollar initial public offering in London, where Internet gambling is legal. That IPO will be the United Kingdom's largest in at least four years, underscoring investor enthusiasm for the $8-billon online gaming market.

PartyPoker marketing director Vikrant Bhargava said he wasn't pleased to learn that many of the poker bot World Series contestants honed their skills on his site, adding that eventually all such cheats get caught. Other sites don't care whether users are human, he said, because the house takes the same percentage of the pot no matter who's playing. But Bhargava said PartyPoker has 100 employees looking for robots, collusion among players and other scams.

Gaming companies won't disclose all their secrets for sniffing out bots, but some of the techniques are simple. Any person playing three tables simultaneously for 48 hours without a bathroom break, for example, or invariably taking exactly one second to bet, is not a person.

Computer gaming experts said the robots have some major hurdles to overcome before they have a chance against the world's top human beings — especially in multi-player games with no betting limit, where the psychology is most important and the number of possible bets is much larger.

Bluffing can be programmed: For every 100 basically worthless hands, for instance, a machine might be instructed to bet heavily five times.

A far bigger issue is the need for abstract pattern recognition. Computers are much worse than humans at anything vague, said poker pro Magriel, a 58-year-old former math professor and world backgammon champion.

At such tasks, "computers are basically idiots," Magriel said. "A computer has an enormous problem recognizing a face. A baby is better."

The need to recognize patterns comes when anyone new sits down at the table. Good poker players learn from the behavior of their foes and adapt on the fly. Computers can store and process millions of past hands, but they have too little data on each new competitor.

For that reason, Schaeffer's team has been focused for years on improving a program's ability to compete one-on-one and learn from as few as 50 hands. After that, the current version does well for a while, until a strong human opponent figures out its patterns. Then the person starts winning.

Magriel once predicted computers would never master backgammon. Now that he knows different, he thinks a better-than- human poker program is inevitable in two or three decades.

"It was a little depressing in chess and backgammon that computers got so good," he said. "In poker, it won't really depress me. I sort of expect it at some point."

[MJ4H]

It amazes me how people didn't learn their lesson when this idea destroyed the game of chess.

[Maple Leafs]

Did it? Chess still seems to be doing fine, thanks. Even the much-hyped Big Blue wasn't especially typical, since Kasparov essentially got spooked by the situation and quit when he didn't necessarily need to (not to mention ongoing rumors that Blue needed some human help).

People assume you can use a computer to do a brute-force solve of these type of games, but they misunderstand how many possibilities there are. You don't have to get many moves into a chess game before the number of possible scenarios nears the number of atoms in the universe -- this isn't tic-tac-toe where there's always a right answer. I'm not sure what the tree looks like for poker, but my guess is that the psychology will further complicate things.

The one bright side is that you could see the sort of changes to poker theory that you saw with backgammon when computers were introduced. I'm amazed at how many people are convinced that current hold-em theory is "right", based on only maybe 30 years of serious thinking.

[MJ4H]

Originally Posted by Maple Leafs

Did it? Chess still seems to be doing fine, thanks.

Yes. I disagree.

Originally Posted by Maple Leafs

Even the much-hyped Big Blue wasn't especially typical, since Kasparov essentially got spooked by the situation and quit when he didn't necessarily need to (not to mention ongoing rumors that Blue needed some human help).

That match isn't central to my point.

Originally Posted by Maple Leafs

People assume you can use a computer to do a brute-force solve of these type of games, but they misunderstand how many possibilities there are. You don't have to get many moves into a chess game before the number of possible scenarios nears the number of atoms in the universe -- this isn't tic-tac-toe where there's always a right answer. I'm not sure what the tree looks like for poker, but my guess is that the psychology will further complicate things.

I don't assume that and I don't misunderstand chess's complexities.


You see here is why I think computers have ruined chess. If you could travel back to a time where there was no computer chess, you would find much more imagination and innovation in the game. Today, all you see is people tweaking their monstrous opening book because fritz tells them that 12. exd4 evaluates to 0.01 higher for white than previously thought.

It's like if perfect chess was 100%, humans used to be I dunno around 20-30%. Computers have brought that number nearer to 80-90, in my opinion. Sounds great, right? Sure if your goal is to play perfect chess, which it has to be if you want to win. But now, everyone plays the same way. It is MUCH more boring to watch and uninteresting. It would be like baseball or soccer played with robots that play nearly perfectly. I don't want to see that. I want to see humans trying, failing, succeeding, innovating, creating. Sigh. It's all gone now. I can't even bear to watch another stupid GM game drawn in 15 moves because no one made a mistake.

Chess is just boring now. Ugh. Of course chess engines have also totally fouled up online play. You can't play a serious game online without computer cheating being suspected. It's gross.

On the subject of poker, what will eventually happen is that computers will be able to implement game theory strategies combined with huge databases of human tendencies tailored to specific situations and combinations of opponents/opponent-types. Great, another game ruined. So much for human innovation in this game, too, I guess.

I think I'm just a purist when it comes to computers playing games with humans. It is MUCH more interesting to watch the humans.

[primelord]

Maple Leafs,

The problem is not that a computer will become the best poker player in the world. The problem is that bots may end up becoming effective enough to beat low and mid limit online games. As I am sure you know, your profit from poker comes from the players at the table that are worse than you. Now you might be able to beat a bot, but if the mass of fish can't then the fish are going to go broke much faster and/or quit the game. Which then just leaves bots and good to excellent players left to play. That is not a very profitable situation.

Bots don't even have to be particularly good to end up ruining online poker.

[MJ4H]

Originally Posted by primelord

Maple Leafs,

The problem is not that a computer will become the best poker player in the world. The problem is that bots may end up becoming effective enough to beat low and mid limit online games. As I am sure you know, your profit from poker comes from the players at the table that are worse than you. Now you might be able to beat a bot, but if the mass of fish can't then the fish are going to go broke much faster and/or quit the game. Which then just leaves bots and good to excellent players left to play. That is not a very profitable situation.

Bots don't even have to be particularly good to end up ruining online poker.

On that point, even the THREAT of "oh I think this table is full of BOTS" will scare the fishies away. While I used to care about that, I don't shed a tear for the online poker communities preying on poor players anymore. I am much more concerned about the purity of the game, myself. I don't want to see another game become a playground for computers that stifles HUMAN creation, innovation, playing-styles, and personality. It is what makes these games great, people. Not playing "perfectly."

sigh

[MJ4H]

By the way, one of the reasons I stopped playing poker altogether is along these lines. All the serious discussionof poker strategy assumes you use Poker Tracker and have these reams and reams of statistics on your opponent's tendencies. I long for the day when it was just one on one. Brain on brain. I really am a purist. I'm sure all the poker sharks will now come rushing to the defense of Poker Tracker because that is what they use to make sure they beat up on all the clueless fishies. I used to do that, too, so I can't complain or really disagree too much. I can just say I got tired of it and realized I miss the game of poker the way it should be.

Just like I miss the game of chess.

[Subby]

There was a bot named Saabpo on Party recently that had very good success in the 20+2 NL sit and go tourneys. The strategy was pretty basic, too - standard push or fold logic - but it could play non-stop, never needed rest, never needed bathroom breaks, never mis-clicked.

Maybe not a worry as a short term opponent, but a monster long term.

[albionmoonlight]

Good points, MJ4H. At least computers have not yet entered the world of juggling.

On topic--what is "illegal" about bots? I know that the private companies can make whatever they want against their terms of use, so it's kind of a pointless question. But its not wrong if I play with Sklansky's books open on my lap. And it is not wrong to use PokerTracker. And it is not wrong to keep an Excel database of people's tendancies. And it is not wrong to use a calculator to compute pot odds. And it is not wrong to have my roommate Chris Ferguson giving me tips while I play. But it is wrong to have a bot play? It just strikes me as a strange line to draw. Data is sent to my computer. Processed. Sent back. I don't look at anyone else's hand. I don't engage in collusion. I just process the data sent to my computer with a tool.

I know that the real reason they are banned is that they scare away the fish. But does anyone have a reason that has some sense of intellectual justification? Why should that tool be considered verbotten when others are OK?

I really like the points that MJ4H makes about how computers are bad for games generally. But once Pandora's Box is open, why ban bots but allow PokerTracker, etc.?

[Airhog]

I dont see the game of chess going down like MJ4h says. Hasn't chess in the 20th century on a whole been like that MJ4h? Then you have players like Fischer who play great chess, but do it with flair and style. Its not enough to win, they want to dominate. Kasparov is very good, and sure computers have probably elevated his game significantly. But this would have eventually happened in chess. Computers just made it happen faster.

[Subby]

For what it is worth, I think PokerTracker on it's own is fine. It is invaluable when used to analyze your own game, spot trends, leaks, etc.. Where I think it possibly crosses the line is when you take 3rd party plug-ins like GameTime+ and PlayerView that pull data from PokerTracker and overlay real-time information about all of your opponents on the poker client. In a vacuum, it simply isn't fair and it enables good players to multitable more effectively (the texture and reads you lose from multi-tabling are somewhat restored with GameTime+ or PlayerView data).

Of course, I use it all the time - it isn't banned by on-line poker rooms and I know my toughest opponents are probably using it. In fact, poker rooms have made it easier to use over the past few months - Party, Crypto, Stars and others all store real time hand histories on your hard drive - and Poker Tracker can be configured to automatically import them. And of course, that information is pulled by GameTime+ or PlayerView and immediately displayed in real time for each opponent.

That said, relying on a mega-small sample size of statistics to get a read on your opponent is not really that bright either. I rarely have more than 50 hands on an opponent at the level at which I play (5/10 sh) and find myself forced to pay attention more. Lots of players have gotten rid of GameTime + and PlayerView because they feel like it has made them lazy with respect to reads.

So, in short - the advantage of these plug-ins is relevant, but probably completely overrated.

[rkmsuf]

not to mention that with PokerTracker and GameTime you actually as the human have to play the game and make all the decisions.

Having a bot do that while you watch I Love Lucy reruns will be the end of online poker when it becomes as accessible as PokerTracker.

[primelord]

Originally Posted by albionmoonlight

I know that the real reason they are banned is that they scare away the fish. But does anyone have a reason that has some sense of intellectual justification? Why should that tool be considered verbotten when others are OK?

Subby already metnioned this, but one of the key things that might allow bots to thrive is that they are immune to fatigue. They are also immune to tilt. They don't care how many times the get sucked out ont hey are just going to keep making the correct mathematical play every time. That is a huge advantage.

Albion you are correct that we have for the most part all of the data that would be fed to a bot at our disposal now. However a bot could do so much more. It could easily keep track of every hand you have shown down and use that in it's calculations to determine what range of hands you might be on. It can track all of the stats that PT does, but in real time and make much faster decisions based off that than we can. It can immediately calculate exact odds, which again with an exact history of showndown hands would be far more accurate than what we come up with.

Some of the advantage is muffled a bit if you are only playing one table, btu where a well written bot would really shine is having an account on all of the party skins and playing 20 tables at once and being able to keep up with all the calculations. There are 2+2ers that sometimes will play 20 tables more for the novelty than anything, but they play strictly ABC poker when they do that. A bot would be able to play an informed competent game over that many tables which is just flat otu somethign a human could not do. That is a huge advantage.

[MJ4H]

Originally Posted by Airhog

I dont see the game of chess going down like MJ4h says. Hasn't chess in the 20th century on a whole been like that MJ4h? Then you have players like Fischer who play great chess, but do it with flair and style. Its not enough to win, they want to dominate. Kasparov is very good, and sure computers have probably elevated his game significantly. But this would have eventually happened in chess. Computers just made it happen faster.

I don't mean the game of chess is headed for an end altogether. But in my opinion it is a mind-numbingly dull thing to try to watch and experience as a spectator. It wasn't always this way. I mean, I *LOVE* chess. I consider myself to be quite a fine player (I have a decent USCF rating). But there is no spark to the game anymore. Years ago, there was. The innovation, creativity, personality, the LIFE is all sucked out of the game. By computers, in my opinion. It is ok with me if others don't share that opinion. I'm not articulate enough to explain exactly why I feel that way. I wish I were because I'm as certain about this as I am anything in the world. Maybe inspiration will strike me and I will be able to explain myself better sometime today. I hope you at least understand WHY I think chess is a worse game because of computers. It is not, as Maple Leafs guessed because I have a misunderstanding of the issue. On the contrary, it is one of the subjects I have spent a lot of time thinking about over the last 10 or so years.

[MJ4H]

Originally Posted by Subby

For what it is worth, I think PokerTracker on it's own is fine. It is invaluable when used to analyze your own game, spot trends, leaks, etc.. Where I think it possibly crosses the line is when you take 3rd party plug-ins like GameTime+ and PlayerView that pull data from PokerTracker and overlay real-time information about all of your opponents on the poker client. In a vacuum, it simply isn't fair and it enables good players to multitable more effectively (the texture and reads you lose from multi-tabling are somewhat restored with GameTime+ or PlayerView data).

I don't disagree with this, really. Poker Tracker can be a good tool to use to study your tendencies. It is a bit dangerous in the slippery slope realm of things, as I think is being evidenced (the GameTime/PlayerView is a proof of concept, I think)

Originally Posted by Subby

Of course, I use it all the time - it isn't banned by on-line poker rooms and I know my toughest opponents are probably using it.

This is the essence of my point. In my opinion the game is worse because you have to have these tools to EVEN COMPETE now. It's like if you gave boxers weapons. The boxers would need the weapons even to compete, but boxing purists surely won't like it. I feel the same way.

[digamma]

Originally Posted by MattJones4Heisman

This is the essence of my point. In my opinion the game is worse because you have to have these tools to EVEN COMPETE now. It's like if you gave boxers weapons. The boxers would need the weapons even to compete, but boxing purists surely won't like it. I feel the same way.

My guess is that there are plenty of poker purists who don't like online play to begin with. If you are about "playing the man," you can still head down to your local B&M casino, have a regular poker night with friends, etc.

I fully understand (and agree with) the arguments regarding fish being scared away, the advantages of a fatigueless/tiltless bot, etc., but I guess I don't get the purist angle. It doesn't strike me that online poker was very "pure" to begin with.

[MJ4H]

I don't have a problem with online play.

[Gary Gorski]

MJ4H - I think I see what you're saying about chess but there's a big difference between chess and poker in that with chess all the pieces are laid out there - in poker the cards are hidden except to the individual holding them. So you could sit and watch chess and realize that a guy is playing "perfect chess" and perhaps even predict what his next move would be but in poker you can't know for sure how a hand is being played until the cards are flipped over in the end. The better players can certainly make pretty good guesses at what an opponent may have or how he is playing the hand but you can't just sit there and see that someone is playing "perfect poker" because sometimes making what would be a horrible statistical play is the right play to make for that particular time unlike in chess where such a move would mean losing the match.

[MJ4H]

I'm aware of the differences between the games. I alluded to how computers would bring poker to an unbelieveably high level above, though--Using game theory and huge databases of opponents/situations. It spells disaster for the game.

Just my opinion, of course.