What do you think of making a second spread sheet using the counting errors method, this time removing the worst game of the WC matches?

In the case of Lasker-Schlechter, it's easy to choose an anomalous game. There is only one choice, game 10, which has 14 errors.

In case of Anand-Kramnik, the game with the most errors is game 9 with five errors. So we should choose this as the game to be excluded.

I have just sent you through email the modified spread sheet minus the worst games.

In this spread sheet minus the worst game, the total accumulated errors are:

Lasker 7 errors out of 9 games, or 0.78/game

Schlechter 8 errors out of 9 games, or 0.89/game

Kramnik 12 errors out of 10 games, or 1.2/game

Anand 8 errors out of 10 games, or 0.8/game

What do you think?

One very obvious fact is that with the worst game removed in each match, suddenly Kramnik is seen as playing a bit more inaccurately than the rest, while Lasker, Schlechter, and Anand are in the same ballpark. However, it does not tell the whole story for the simple reason that Lasker and Schlechter did produce an error laden game of a technically much worse kind than Kramnik ever did.

It's a conundrum. Perhaps it's best to simply state the conclusion as:

With the worst game of each match excluded, Lasker, Schlechter, and Anand played with errors per game ranging from 0.78/game to 0.89/game, very close to each other in accuracy, while Kramnik played more inaccurately with 1.2/game. Caveat: However, in the excluded worst game of his match, Kramnik played more accurately than both Lasker and Schlechter in the excluded worst game of their match.

In addition:

I see foresee a possible minor problem in the option for the excluded worst game.

Anand-Kramnik has four games with four errors - games 2, 3, 4, 10. Suppose game 9 also happened to have 4 errors, which would mean five games with the same number of errors, which game should be excluded?

I have been thinking it should be the game with the errors most evenly spread between the two players.

These are very similar figures to those that I posted on the Anand page, and which you copied back to this page on 24 October.

The main difference is that in compiling adjusted averages for the players, I've removed the most error-ridden game played by each player, rather than merely the most error-ridden game.

Game 10 of L-S has the most errors for both players, however, as you point out the errors are more even distributed through the A-K match.

Statistically, if you want to measure errors per game for the World Championship match, inclusive of both players, then of course A-K game 9 is the game that would be omitted to smooth the figures the way omitting L-S game 10 smooths out the main statistical "bump" in that match.

However, once we look at the stats for each individual player, and use the same statistical smoothing process, then we should <omit the game which features each player's worst figures>.

In terms of raw errors (all errors of 0.60 or greater lumped together), then Kramnik's 4 mistakes in game 9 would be omitted. In the case of Anand, game 10 would need to be omitted.

Because we cannot weight the errors for reasons already discussed, disaggregating error statistics has to be done carefully. The easy way to do it, that's enabled by using bandwidths, is to aggregate them from the top, eg

- blunders

- blunders and 1.20-1.40 mistakes

- blunders and mistakes made in the two bands in the 1.00-1.40 range

all the way down to 0.60.

Generally they don't, as you can see if you observe carefully all available researches. A very important fact one must know is that the results show biases towards positional players, the simpler positions a player prefers, the more overrated his results in such analyses tend to be as long as practical play is not taken into consideration in the analysis methodology.

Another way how to make older players to look overrated is not to take difficulty of positions and time controls into account and look at positional players only. That's precisely what <Bridgeburner> has been doing. No wonder it easily leads to faulty conclusions.

<But whether ply search (Guid/Bratko) or a fixed complexity data/line of Sullivan (am I correct here?) is sufficient to determine the complexity of the games I wonder.>

It is sufficient. Take a look at the complexity vs average error graphs in their studies. You can see yourself that more difficult positions induce less accurate play.

<Chess did progress in 100 years I suppose.>

Exactly.

<Like Carlsen who grew up in this computer age. Anyway keep up the good work and looking forward to see next results.>

I analyzed Carlsens games in Nanjing 2009. Without taking into accont how much practical play was involved in his games, the conculsion I reached to was that his performance according to the accuracy of play was 2970 - quite close to ELO rating performance.

I know very well what the popular image of him is, but often what people perceive and what the reality is do not match. Take morphy for instance, he's widely regarded as a wild tactical attacking palyer, but in reality his play was rather conservative and solid in comparison with leading romantic players of his era.

Although Bratko & Guid rate his play slightly more complicated than on average, according to Sullivan and my onw research his positions were rather easier than on avergae.

<Still in the Sullivan's ranking he is tied 4 in the three year peak (no draws included)! >

I suggest looking at results with draws included. There's no objective reason to eliminate them in the conclusions.

http://www.truechess.com/web/champs...

Lasker's average rank across all tables is 10.5 out of 16.

<So how does that Carlsen 2970 performance compare.>

Elo ratings as of 2008 were taken as a basis of comparison.

There is a reason why <Bridgeburner> and I decided a long time ago to choose World Championship Match games for this project.

I tend to use the terms <difficulty> or <complicated> instead of <complexity> because the latter is technically a term to denote for a structure an increasing number of component parts and increasing amount of interactions for these components. In which case, the most complex positions would be early opening positions, and the least complex would be endgame positions.

What you are talking about is really <difficulty>, a subjective human evaluation of a position. There are positions which to most human chess eyes look <simple>, meaning they fit into the usual positional patterns that we can subjectively easily evaluate and analyze. There are also bizarre or messy positions which may be just as complex as simple ones with the same number of pieces, but are difficult for humans to evaluate and analyze because they are not as amenable to our usual ability for pattern recognition.

I have posted above my objections for the use of <difficulty> (what you term as <complex>) as a parameter of chess strength.

<Chess did progress in 100 years I suppose.> In opening theory, chess did, and it does all the time. However, outside openings theory, a player has to play over the board with his innate talent and acquired experience and training; and these have human limits. (Please see one of my posts above on this.)

In each era, there is an opening theory that is generally accessible to all the top masters. Thus the advantages of any new development tend to cancel out in competitive play in each era, because each top master of each era has access to the same opening theory and research methods. This would mean that openings in the Lasker-Schlechter World Championship Match (1910) would tend to be inferior to 21st century openings. However, once they get into middlegames and are relying on their native talent, experience, and motivation, the Project indicates that Lasker and Schlechter played the games of their match mostly as well as Kramnik and Anand did.

<I think it is interesting also to compare how people who are trained by computers will do! Like Carlsen who grew up in this computer age.>

<difficulty> and <complexity> are nebulous concepts, mainly subjective as <visayanbraindoctor> has taken pains to point out, and in any case are extremely difficult to quantify and even more difficult to use to operate upon error figures.

Also <nimh> emphasizes the importance of time controls in calculating accuracy. His thesis is that the more relaxed time controls of the old days allowed for more considered moves, and fewer apparent errors.

Nevertheless, compare the old masters with the modern players (check the CG.com database) and you'll find modern masters play a much higher proportion of draws than masters a hundred years ago. Moreover, the game with easily the most errors (nearly half the errors in the match) was the famous and decisive game 10 of the Lasker-Schlecter match which was played over three days, and therefore at least two adjournments, which theoretically should have provided the players considerable extra time for analysis. All this leads me to believe that whatever impact the slightly different time controls had cannot be reliably factored in, especially when hedge factors such as arbitrarily determining that all adjournments simply add 1 hour to a game are thrown into the mix.

The bottom line with my research is that the data is as accurate as it is feasible to acquire, and will be even more accurate with the upgrade in hardware and software which I will bring to task in the next phase of my project. <I cannot over-emphasize the important of <<data integrity>>. All the wizardry of data management is worthless without the data, or without data in which one can have confidence.>

Ensuring that the engine evaluations were as accurate as possible took about 95% of the period of over a year in which it took to analyse the 21 games in the 1910 and 2008 World Championship matches, whereas other studies have hundreds, if not thousands of games under their scrutiny. I don't believe that the fidelity of the error data of some of these projects is anywhere near as reliable as they should be, or that the moves have been sufficiently checked for accuracy.

<The primary unit of measure with these studies is the difference between an engine's evaluations of its preferred moves and its evaluations of the moves actually played. This basic brick of data must be unquestionably accurate, as the difference - what <nimh> refers to as the <average error> - is of <little or no value> if the engine's preferred move is wrong.>

Typically, a move in one of these studies, eg: <nimh>'s, might be subject to 5 minutes of engine time, or 15 ply...this may be sufficient for a quick evaluation of games of FM standard or below, but super-GMs and world champions are another kettle of fish altogether.

Complexity is an issue in my work insofar as it determines how long it takes my engine to calculate evaluations that are entirely consistent with every other evaluation in the game beginning from, or near, the end of a theoretical line.

Also, I examine each and every move in depth, including openings and endings through to the endgame. I don't believe in skipping the opening because it's established theory, and nor - more importantly - do I believe in omitting endgames because some engines may not be good enough to properly evaluate them. In my opinion, this is a complete cop out as many crucial errors occur in the endgame.

This may not even be true in World Championship matches between Lasker and Schelecter, Lasker and Capablanca, and Alekhine and Capablanca, the three best players of the pre-WW2 era. There is no weakie to beat up between these players. (",) As seen in the Lucena page, it would be reasonable to think that these players, who played with very little error, would allow only for very few game losses in an event that required them to give it their best shot for them to win, and so matches between them would tend to have a lot of draws.

Regarding modern super GM tournaments, not one of them is typically as strong as the typical World Championship match. Even super GM tournaments will feature relatively weaker players even if they are all very strong. A World Championship match would feature no relatively weaker players at all.

Let us take the recently concluded Bilbao chess tournament, allegedly the strongest chess tournament in history. The winner, Kramnik has to play two games each against Anand, Carlsen, and Shirov, for a total of 6 games. That obviously is weaker opposition than playing Anand 11 straight games as he did in the 2008 WC match.

I think this is actually quite an important point. On average, I would expect draws to be of higher quality than decisive games (simply because it takes at least one serious error to make a game decisive). Around 60% of modern grandmaster games end in draws, whereas only around 20% of 1900 era games ended in draws. Thus, excluding them will differentially bias the results in each case.

Welcome back. I can't see why anyone would exclude draws!

<<data management is <<even more important>> than the data>>

In the Scientific Method, this is totally unacceptable.

The hypothesis is supported (or proven false) by empirical data that is gathered or generated by an experiment with a replicable methodology. The management of data is totally wasted if the method of gathering or generating the data is vague or inaccurate. First, the data has to be valid.

<<If the data is invalid, management of that <data> is <<absolutely irrelevant>>.> You may as well make it all up.>

My hypothesis is it's because tournaments in the 1900s had a lot of weakies in them, who made a lot of errors, and who may have even affected the play of the really strong players. This is why the subject of the Bridgeburner Project is World Championship matches, wherein there are no weakies to beat up, or for a strong master to get complacent about. I have a post on this above.

I use <weighted average error>. This is the sum of errors divided by total number of moves.

And since i'm using a cap of 1.1 (i adjusted from 1.00), 1 error does not drastically reduce the accuracy value of a player.

Example:
Player makes 9 perfect moves and 1 blunder capped at 1.1. The average error will be 1.1/10 or 0.11

I introduce a <accuracy percentage> value.

The formula is:
Accuracy Percentage = No.of moves/(No. of Moves+sum of errors)

In the above example:
AP = 10/(10+1.1)
so that
AP = 90.1%

I also count all the errors values (capped at 1.1).

In moves 23, 25 and 29 the errors were capped to 1.1. Move 29 is mate in 10.

This is because as (someone pointed out earlier) this has an accumulative effect. I believe this system (of counting the smallest errors) is more mathematical and scientific.

Somebody might ask, "why then you cap at 1.1?".

2 reasons.
Mathematics has to deal with inflation at the end of the spectrum or even "infinities" and must apply "renormalization", hence a "cap". This is also being applied in the elo system.

By the way, not only the opening progresses. We get more strategic knowledge, new patterns etc. It's not like top players' strategy didn't change since "My system". It did (didn't Kasparov once say that in their first match with Karpov there were nuances which back then were <understood> by nobody but the two Ks?). And the better understood strategy is, the better the games are...

There are potential biases with regard to draw percentages in the older games.

Game databases might be biased toward decisive games as we go back in time. I know I'm having trouble finding more than half the games from the 8th American Chess Congress in Atlantic City 1921.

Draws in early tournaments often did not count, and had to be replayed. This might or might not be shown in the crosstable, which could underestimate draws if that is a data collection source.

As a quick example, I've recently done a game collection on the 7th American Chess Congress in St. Louis 1904 (Game Collection: St. Louis 1904). A quick look at the crosstable usually presented suggests only <two> draws in the nine rounds played. However, a drawn game required a replayed game with colors reversed. If the second game was also drawn, only then would the result be scored as a <single> draw. If you go through all the individual games, there were actually <twelve> draws.

Other tournaments had different rules for what to do with draws. This could influence players to press harder for wins.