< Earlier Kibitzing · PAGE 8 OF 13 ·
Later Kibitzing> |
Aug-02-09 | | Everett: Thank you <nimh> and <Bridgeburner> for your work and thoughtful posts. I was wondering if either of you can help <tamar> and me understand the complexity score as discussed on the truechess site by Sullivan. In an analysis of 144 games from the Karpov-Kasparov matches, Kasparov is given a much higher complexity score, though they are of course playing nearly identical positions. Thanks in advance for your help. Also, can someone explain why "complexity" is needed for these studies? I guess I don't see how it can be defined objectively, nor applied accurately. |
|
Aug-03-09 | | Bridgeburner: <Everett>
I'm not sure whether you saw Charles Sullivan's FAQ in which he addressed the question in respect of Kasparov and Karpov, so here’s the link in case you didn’t: http://www.truechess.com/web/champs... The difference between the engine’s evaluation of the move played and the move it replaces during the iterations doesn’t make sense to me…I can’t see why he doesn’t calculate the difference between the old move’s value before it is replaced and the value of the new move at its final iteration rather than initial iteration. I'd think that the former is closer to the move’s “true” value. I personally don't see the point of complexity calculations, especially as Sullivan concedes that complexity is also accompanied by errors, and I’m also dubious that finding the moves with lower complexity values that seem to come with being on the defensive is any easier for a player to find except where they are part of a forcing sequence. But the ones that use these measures - <nimh> and User: CharlesSullivan, can speak for themselves. |
|
Aug-03-09 | | nimh: <Also, can someone explain why "complexity" is needed for these studies? I guess I don't see how it can be defined objectively, nor applied accurately.> It's needed for determining which positions are more difficult to play. Although it is impossible to define it finitely and reach a consensus which way is the best, it clearly shows that higher complexity scores are accompanied by higher average errors. There is a relation and it would be an insanity not to take it into account.
An assumption that, in every chess position, finding best moves and playing at higher accuracy is equally difficult is plainly wrong. <The difference between the engine’s evaluation of the move played and the move it replaces during the iterations doesn’t make sense to me…> I'm using practically the same formula as B&G did, to me it is good enough. |
|
Aug-03-09 | | nimh: BTW, the reason why the differences during the iterations were taken into account is well explained in paragraph 2.3: http://www.chessbase.com/news/2006/... <The di®erence between the evaluations of the best and the second best move
represents the signi¯cance of change in the best move when search depth is
increased. It is reasonable to assume that a position is of higher complexity, that
is more di±cult to make a decision on a move, when larger changes regarding
the best move are detected when increasing search depth. Merely counting the
number of changes of the best move at di®erent search depths would give an
inadequate metric, because making a good decision should not be di±cult in
positions where several equally good choices arise.> |
|
Aug-03-09 | | Everett: So, it seems that players are awarded for putting themselves in difficult situations, and then blundering when they can't figure it out? ... or they are put into this situation by the other player, thereby somehow being given allowances because the situation was made difficult for them? I don't think complexity works, because:
<It assumes the person who is making it more complex is the person who is in the complex position.> and <it assumes complexity means the same thing to humans and programs alike.> Further, many great defensive moves are not "only" moves. Both aggressive and defensive moves can be forcing or flexible. I don't see how one type is invariably more forced than the other. Seems to depend on position. We all know programs and humans understand positions differently, and it is quite often that computers see the best move clearly through massive complications while a human would struggle with it. In such a situation, there would be no complexity of note to the computer but quite the contrary to the human, yet the human would be "punished" for blundering in such a position. Finally, many positions don't offer themselves up to the interpretation of a "best move." There may be a few ways to go, and a person simply has to choose which way to go and honor the characteristics of the ensuing positions. |
|
Aug-03-09 | | percyblakeney: <it seems that players are awarded for putting themselves in difficult situations, and then blundering when they can't figure it out? ... or they are put into this situation by the other player, thereby somehow being given allowances because the situation was made difficult for them? I don't think complexity works>
There is a problem with or without the complexity thing though, take for example Ulf Andersson's games from the last decade. Most of them are totally mistake free, rather short draws where no player tries very hard to win. Here's one of the longer examples: Vu Rasulov vs Ulf Andersson, 2009
It can be compared with this game, where Topalov is pressing Ivanchuk hard in a top game, and in time trouble Ivanchuk goes wrong in a difficult position: Topalov vs Ivanchuk, 2009
In a way the first is a better game because both players mainly seem interested in exchanging down to a draw, but it does feel as if Topalov and Ivanchuk played much better chess in their game regardless of Ivanchuk's time trouble mistakes in the end. |
|
Aug-03-09 | | visayanbraindoctor: <Everett> I mostly agree with you, although I may use different words. Please see my posts in <Bridgeburner's> forum. <There is a problem with or without the complexity thing though, take for example Ulf Andersson's games from the last decade. Most of them are totally mistake free, rather short draws where no player tries very hard to win. Here's one of the longer examples:Vu Rasulov vs Ulf Andersson, 2009>
This looks to me like a Grandmaster draw, although more prolonged than the usual. Such games are often characterized by both players following opening variations that are already known to lead to drawish positions, and often embark on a policy of liquidating their pieces whenever it looks safe. Two masters can play such games a thousand times over, making no mistakes whatsoever, but they don't really mean a thing. |
|
Aug-03-09 | | nimh: For some reason, I think fans of Tal or Kasparov would have liked the difficulty paramaters and denounced attempts to calculate one's accuracy on the basis of raw values :) |
|
Aug-03-09 | | Everett: One more word on "complexity," as I find it a very interesting (read: impossible) thing to measure objectively. <Complexity, as I have designed it, is higher for "aggressive" players who must justify their aggression by finding deeper, "non-obvious" moves & combinations and lower for "passive" players who are content to keep the position balanced in which there are often several perfectly acceptable "positional" moves that do not radically disturb the balance. Note: when a player is thrown on the defensive, his move choice is usually greatly restricted since he is often "forced" to make a particular (often fairly obvious) move or immediately lose the game. This fact helps explain why the losing side almost always has a lower complexity score than the winning side (since a losing player usually must play more defense than offense). > Is how Charles Sullivan describes his design of complexity. Now, assuming his statement is true (though I disagree on a few points), the aggressor would have a tough time choosing the best continuation, but since the defender's choices are often "forced" the aggressor can depend on the answer, thus making the prediction of his opponent's move easier. Conversely, the defender must consider all the different attacking permutations that can occur from the aggressor, thus making his response harder to divine before the aggressor plays his move. That's a lot of pressure on the defender! I feel there is a moralizing going on here, a bias for aggressive play that motivates the creation of this abstract "complexity" score to justify inaccuracies. <Perceyblakeney> I agree with your post. Despite the hyper-accuracy of current programs, I feel any assessment fails to capture the human struggle of chess (though I enjoy watching the attempts!) |
|
Aug-03-09 | | Ed Trice: Tal's quote sum up "complexity" from your discussion nicely: "You have to take your opponent into a deep, dark forest where 2 + 2 = 5, and the path out is only wide enough for one." |
|
Aug-03-09 | | Bridgeburner: The issue about risk free, mistake-free GM draws is a valid one of course, undeniably so, but fortunately for all of us including the likes of us game mappers, most games are not like that. It's the struggle that enables the quantitative method to work, but the quantitative method will never <capture the human struggle of chess> any more than dissecting a plant or an animal will capture the essence of its existence. The method is not looking for "better', it's looking for "accurate". I'll only be examining WC matches, as the percentage of GM draws is much lower. But I'll also stop to smell the flowers. |
|
Aug-03-09
 | | tamar: <nimh: For some reason, I think fans of Tal or Kasparov would have liked the difficulty paramaters and denounced attempts to calculate one's accuracy on the basis of raw values :)> Why? I like those players, but they don't need any help. The real problem with complexity based on anything other than each individual position is that you introduce a bias into the study. I still haven't heard an explanation how Karpov could score low complexity and Kasparov high complexity based solely on the matches they played against each other! It makes no logical sense. |
|
Aug-03-09 | | nimh: <Why? I like those players, but they don't need any help.> Actually they do, because, like other attacking wild-styled players, the positions they played in were quite messy and therefore difficult to play accurately. Any study that fails to consider the importance of difficulties of play in various positions, is what I call biased. Moreover, the complexity isn't the only parameter of difficulty. For the sake of objectivity, one must take into account as many as possible of them. |
|
Aug-03-09 | | Everett: <It's the struggle that enables the quantitative method to work, but the quantitative method will never <capture the human struggle of chess> any more than dissecting a plant or an animal will capture the essence of its existence. > Well put <Bridgeburner> <nimh> If positions are messy for the "attacker," they are messy for the "defender" as well. This is the crux of Tamar's (and my) argument. Complexity cuts both ways, then the only thing the attacker should be given is credit for creating such a mess, not an allowance for making errors in the ensuing chaos. And who should be given credit for making the games messy in the case of Kasparov and Karpov? One can believe that Kasparov was "forced" to make things sharp because this was the only way to beat Karpov. So, it is Karpov's solidity and strength in quieter positions that forces Kasparov to make things messy, and they both make mistakes because of it. |
|
Aug-04-09 | | nimh: <If positions are messy for the "attacker," they are messy for the "defender" as well. This is the crux of Tamar's (and my) argument. Complexity cuts both ways, then the only thing the attacker should be given is credit for creating such a mess, not an allowance for making errors in the ensuing chaos.>
I know that it work in both ways. In the case of my study, it doesn't matter actually which side is guilty of creating chaos.
If one side makes positions messy, his and his opponent's complexity parameter rises and therefore his mistakes are rewarded. If one doesn't give him an allowance, you won't get correct results as well. <And who should be given credit for making the games messy in the case of Kasparov and Karpov? One can believe that Kasparov was "forced" to make things sharp because this was the only way to beat Karpov. So, it is Karpov's solidity and strength in quieter positions that forces Kasparov to make things messy, and they both make mistakes because of it.> Not true, Kasparov didn't play wildly because of Karpov, but rather his own preferences. It's also true that against Karpov he usually played much more calmly. |
|
Aug-04-09 | | nimh: <So, it seems that players are awarded for putting themselves in difficult situations, and then blundering when they can't figure it out?
... or they are put into this situation by the other player, thereby somehow being given allowances because the situation was made difficult for them? > The only thing they're awarded for is being in a situation where pölaying acurate is complicated. The rest you say is meaningless. <I don't think complexity works, because:
<It assumes the person who is making it more complex is the person who is in the complex position.> and <it assumes complexity means the same thing to humans and programs alike.>> See the fourth graph
http://www.chessbase.com/newsdetail...
the first graph
http://www.truechess.com/web/champs...
on page 10
http://web.zone.ee/chessanalysis/su...
What dou you think they indicate, if not how one's accuracy gets worse in difficult positions?
And no one has assumed that it means the same thing for either humans and programs - in fact, humans are more affected. <Further, many great defensive moves are not "only" moves. Both aggressive and defensive moves can be forcing or flexible. I don't see how one type is invariably more forced than the other. Seems to depend on position.> Making a distinction between defensive and attacking moves is not necessary. The 'forcedness' is covered by another parameter - 'difference'.
The larger the difference engine's suggestions #1 and #2 is, the more important and easier it is to pick the best move and, surprisingly, the overall accuracy decreases too.
Complexity has nothing to do with it, and it is indeed true that defensive moves are not necessarily of more 'forced' nature. <We all know programs and humans understand positions differently, and it is quite often that computers see the best move clearly through massive complications while a human would struggle with it. In such a situation, there would be no complexity of note to the computer but quite the contrary to the human, yet the human would be "punished" for blundering in such a position.> Not true. No computer, be it however good tactician, would be able to see everything after 2 or 3 plies.
Read more carefully, how complexity is calculated. All depths from 2 to 12-14 are taken into account. Humans suffer <Finally, many positions don't offer themselves up to the interpretation of a "best move." There may be a few ways to go, and a person simply has to choose which way to go and honor the characteristics of the ensuing positions.> That's why I use other parameters of difficulty. The complexity alone is not sufficient for indicating difficulties in play. And I agree that the notion of 'best move' is futile in many positions. <There is a problem with or without the complexity thing though, take for example Ulf Andersson's games from the last decade. Most of them are totally mistake free, rather short draws where no player tries very hard to win. Here's one of the longer examples:> Here, it must be pointed out that the reason why you think the latter game is better is that it inculdes higher-rated players :) While I see that the 'absolute (or raw) average error' value of the former game is lower, it is also evident that the latter game is more difficult and all criterions of difficulty are higher, thereby most likely resulting in a <lower> 'excpecter error' value.
It's funny how you say it might be a problem even with complexity thing, whereas it actually is exactly the way how to eliminate bias. |
|
Aug-04-09 | | nimh: <This looks to me like a Grandmaster draw, although more prolonged than the usual. Such games are often characterized by both players following opening variations that are already known to lead to drawish positions, and often embark on a policy of liquidating their pieces whenever it looks safe.> Most of Capablanca's games look like this. Easy to play accurately and avoid blunders. No wonder his 'absulute error' value is so low and not reflecting the reality about his playing strength. BTW, this isn't a grandmaster draw, you know. To claim that Capa's simple and clean drawish games are genuine and a sign of his super-hyper-mega-maxi-ultra-accuracy while denouncing similar games by modern players under the pretext of being a 'prolonged gm draw' or 'not signing will to play for win' is an act of hypocrisy! <Two masters can play such games a thousand times over, making no mistakes whatsoever, but they don't really mean a thing.> Capa aslo played countless similar 'mistake-less' games that don't mean a thing. <The issue about risk free, mistake-free GM draws is a valid one of course, undeniably so, but fortunately for all of us including the likes of us game mappers, most games are not like that.> If you feel it is unfortunate for you, most likely you don't have a good methodology yet. <The method is not looking for "better', it's looking for "accurate". I'll only be examining WC matches, as the percentage of GM draws is much lower.> Accuracy and the level of play are connected without doubt. It's needless to confine oneself with WC matches only, just do not take short draws into account. |
|
Aug-04-09 | | percyblakeney: I wonder how much the results of the Bratko/Guid study have "changed" after the Anand-Kramnik match. They only evaluated title matches, and Kramnik may have dropped a place or two after Bonn, while Anand (not included in the study) should score a decent result. It is interesting that their analysis with Crafty ranks Deep Blue 1997 as playing much better than any human before or after (for example Kramnik in London 2000), but in the Rybka forum many still evaluate the engine at 2700 Elo. |
|
Aug-04-09 | | visayanbraindoctor: <nimh> I know that you think that <difficulty> can be quantified. I don't. I have posted my reason above. Here is a post of mine in <Bridgeburner's> forum to the same effect. Regarding <difficulty>: <Bridgeburner: I don't think the parameter is valid> Obviously I agree with you. When we judge anything to be difficult, our perceptions affect our judgment. My problem with this parameter is very fundamental, and comes directly from my line of work. Time and time again, I have seen patients describing almost the same phenomenon differently, if human perception enters the description. Neuro people naturally invent scales in order to measure phenomena that may be affected by differing human perceptions and their subsequent reactions, but such scales are never perfect for the simple reason that human perceptions and their subsequent reactions differ from individual to individual. Another medical personal may call you and tell you that a patient's score in some scale (say a scale trying to measure sensorium) is so and so, but in the end, if you want to get the most reliable assessments of the patient, you better come and see the patient for yourself. If such scales were completely rigidly defined and objective, then it should be possible to just phone in a score, and it should describe perfectly the patient's condition. It helps of course, but in the end you should always come and see the patient for yourself in order to assess him as properly as possible. |
|
Aug-04-09 | | visayanbraindoctor: <nimh: If you (Bridgeburner) feel it is unfortunate for you, most likely you don't have a good methodology yet.> I don't see why. <Bridgeburner> has a good methodology and there ARE Grandmaster draws. They may be difficult to quantify but they do exist. They may be difficult to quantify for they involve subjective human decisions, perceptions, and reactions. IMO they basically depend on the decision of the White player to play for a draw right in the opening. The Black player may think he perceives it, and he might react by going along with it, or he may not. If the Black player goes along with it, they usually end up entering an opening variation that is known to lead to a drawish position. Here are posts of mine in <Bridgeburner's> forum: The phenomenon of Grandmaster draws has existed for a long time. Aside from the factors you stated above, it could be related to the concept of <chivalry> among pre-WW2 masters. Capablanca and Schlecter for instance have stories regarding chivalrous behavior that seems utterly absent in today's international tournaments. In brief, I think that it was widely deemed as the height of bad manners for two chess players to regularly do GM draws before WW2. Realistically speaking, if two masters would like a GM draw before WW2, nothing could prevent them; for instance as in game 3 of the Lasker-Schlechter World Championship Match (1910). What kept most of them from doing it regularly probably had something more to do with such draws looking unchivalrous. Schlecter for instance got hit with the label of drawing master, and during his match with Lasker, he could have played more aggressively precisely to prove to the world he wasn't being an unchivalrous coward. <if they were putting up the money themselves, then short GM draws become quite pointless, as at least one of the players would be diddling themselves.> I was wondering at this. Apparently, many of the matches pre-WW2 were personal challenges. If one master had an issue as to who was better than the other, they simply made a bet and played a match. Aside from the money that they raised, it seems personal honor was also at stake. The matches looked like two medieval European Knights jousting. It would be funny if they started their horses running, spears pointing at each other, and then stop in the middle of the field to shake hands. Since the 1970s, it seems that if one player wants to say "I am better than you", he just points out his higher Elo rating. Personally, I prefer the medieval joust. (",) I believe that in the WC cycles (for example in zonal tournaments) one can also see countrymen making GM draws with each other if they think it would help them secure better chances of qualification over players from other countries. For good or for bad, this seems to be quite an ingrained practice. <Bridgeburner: It sent Fischer ballistic when he saw the Soviets doing it. Sometimes the more things change, the more they seem to stay the same.> <Bridgeburner: I guess the huge increase in theory has just enabled GM draws to get longer, while other factors make them more frequent.> I have seen GM draws wherein both sides played theory for more than 20 moves in just a few minutes, diddle around in a drawish endgame for another 20 moves, and then draw. Others just agree to a draw in less than 10 moves. It's their decision.. Like anything involving human perception, decision, and reaction there are degrees of Grandmaster draws. In the less-than-10-move-draw, or in games that follow opening theory known to both masters that lead to a perpetual, the games are obviously very much of Grandmaster draws, and many of such games are pre-arranged, while in others both players probably take up the decision to draw very early. In other games, both players probably make the decision a while later; but for some reason they do, and when they do, pieces usually come flying off the board very quickly without a real fight. You might ask how can we rigidly define this type of Grandmaster draw? We can't as I have said above. It's a gray area, like so many things human. |
|
Aug-04-09 | | nimh: You're misinterpeting my point. I've never denied the existence of GM draws. What I said is that Rasulov vs Andersson is not a GM draw in my opinion. |
|
Aug-04-09 | | Everett: <nimh> I very much appreciate your patient responses. Can you explain how this sentences escapes subjective view, from your first link, 4th paragraph above <A method to assess the difficulty of positions was designed, in order to take into account the differences in players' styles and to compensate for the fact that calm positional players in their typical games have less chance to commit gross tactical errors than aggressive tactical players.> Really? Is it easier for the "calm positional players" to keep a nagging advantage with precise moves? Many think it's the hardest thing to do in chess, which is why the great stylists of this sort are often the most respected players. This is a strange deference to aggressive players. I think a better study would be to see how aggressive players make their opponents grossly blunder, for this is the root of their style; to apply pressure in ever inscrutable positions in the hopes of making their opponents choose inferior continuations. Somehow, it seems, complexity tries to allow the aggressive players to get all the benefit but none of the risk (much like many capitalists would like things). Players should simply be given responsibility for the crazy or simple positions that they produce, allow, and navigate through game play. By creating a "complexity" score, one undermines this responsibility. |
|
Aug-05-09 | | nimh: <Is it easier for the "calm positional players" to keep a nagging advantage with precise moves? Many think it's the hardest thing to do in chess, which is why the great stylists of this sort are often the most respected players.> Yes, it is. Although it's rather about maintaining accuracy of their play, than keeping advantage.
I don't think it is harder, positionally playing players usually have more positions where there are more virtually equal choices and less tactical surprises. They're respected because they usually have larger knowledge and are less inclined to take unnecessary risks, diminishing the chances to lose. How would you calculate complexity values? And how would you take into account concepts like 'making their opponents grossly blunder', 'risk', 'responsibility'? Is it even possible? |
|
Aug-05-09 | | Everett: <nimh> Thanks for answering again. I think you, among others, have done a great service in discovering that a more "complex" position results in more blunders, on both sides of the board. I don't know how one would do it, but it would be interesting to see if there is a correlation between aggressive players and the propensity for their OPPONENTS to blunder, and when it occurs. This is obviously a less ambitious study than the "who is better" ones we're debating, but I think it might provide some evidence of the success of some players. I agree with your sentiment that it is impossible to assess the ideas of "pressure created on opponent," "risk" and "responsibility," because it involves, at least partly, getting into an player's mind. I'm not sure I WOULD calculate complexity values: it seems problematic for the reasons in my last post. I also must say that I am not well-versed in all the different ways it can be done, though I think the number of ply performed before the computer realizes the "best" move is a good start. Here would be an example of a complex game, and <bridgeburner> at the bottom describes why. Kramnik vs Morozevich, 2007
Yet, I hesitate, as it seems that if Kramnik saw all the other variation clearly as losing, and saw only something murky with his 12th and 13th move, was it truly complex for him, or just practical? He certainly admits to not seeing all the consequences of the moves played until four moves into the combination... Another one of those gray areas re: complexity... All that said, I appreciate all the work your doing <nimh>. I reserve the right to stand corrected, and I may have the whole complexity issue wrong... |
|
Oct-30-09 | | Whitehat1963: So, here's a question that kind of relates to the rise of Elo ratings. It seems that people's IQs are higher today than they were 100 years ago. The tests are much harder today. (So I've read anyway.) Does that mean people are smarter today than they were 100 years ago? If that stands to reason, what does that say about the intelligence of someone like Shakespeare or the musical prodigy of Mozart or the incredible intelligence of a John Stuart Mill or Voltaire? Were these guys about as intelligent as today's average man? |
|
 |
 |
< Earlier Kibitzing · PAGE 8 OF 13 ·
Later Kibitzing> |
|
|
|