< Earlier Kibitzing · PAGE 198 OF 242 ·
|Oct-21-10|| ||frogbert: <I checked the dates before my last reply, and there was not enough consistent evidence from the numbers about mondays.>|
the list "ids" (dates) don't tell when the update was made, it tells when the games that were rated were <played>: on or before that date.
several dates for which there exist lists were actually published the next day. i happen to have actual upload time for every list, and more often than not the date of publication is different from the list date.
latest example: the lists of october 18th and 19th were published october 20th.
details aside: the simple point is that you have way too little information available to tell others <why> updates are made when they are made. that's what <you> need to admit, the sooner the better.
<If what you have admitted is true, then there should exist a correlation between the quicker site updates, and Carlsen's positive results. >
that would simply go to show that studies based on correlating events in order to establish causal relationships are littered with pitfalls that so-called serious scientists fall into all the time.
the method is again flawed if your argument goes like this:
"in p% of the total "quick updates" carlsen's result was positive, and in r% of the total "quick updates" carlsen's result was negative, and p > r, ergo carlsen having positive results causes quicker updates"
what's the most fundamental requirement for a <causal relationship>, bartonlaos? the <cause> needs to be the <reason> that the <effect> can be observed.
if you consider <percentages of the quick updates> you start at the wrong end of things. "if p1 then q". "if p2 then q". you can't make an argument about p1 and p2 by studying the occurences of q, arguing backwards.
if carlsen had 10 results, 7 positive and 3 negative ones, and updates were immediate after <all> 3 negative ones, but only immediate after 4 of 7 positive ones, we have p > r. but immediate updates happened in 100% of the cases with negative results and only in 4/7 of the cases with positive results. going on to conclude that positive results causes faster updates would be worthy of some kind of "medal".
|Oct-22-10|| ||SwitchingQuylthulg: <frogbert> I made a quick comparison between a very early live rating list (Nov-18-07) and yesterday's list. Several changes were very clear; most notably, the number of players at 2700 or above has increased dramatically from 23 to 39 (there are now more players above 2720 than there were above 2700 less than three years ago). What's your hypothesis as to why? Do you expect this development to continue in the next few years?|
|Oct-22-10|| ||bharatiy: on same line are you considering limiting your list to top 20 or 25 instead of rating number. That may mean more consistent list and less work for you. Also in one year time you may as well do a list of 2750+ players and have 20-25 in them.|
|Oct-22-10|| ||slomarko: <Several changes were very clear; most notably, the number of players at 2700 or above has increased dramatically from 23 to 39 (there are now more players above 2720 than there were above 2700 less than three years ago). What's your hypothesis as to why?>|
inflation. and it's not just a hypothesis, it's a fact.
|Oct-22-10|| ||alexmagnus: Development of the number of live rating players (each change):|
26.04.08: 28 (new max)
03.05.08: 29 (new max)
02.06.08: 30 (new max)
06.08.08: 31 (new max)
19.08.08: 32 (new max)
24.08.08: 33 (new max)
02.11.08: 34 (new max)
04.10.09: 35 (new max)
08.10.09: 36 (new max)
2010 still to come...
|Oct-22-10|| ||alexmagnus: Note the lack of new maxima between 2.11.08 and 4.10.09. Even more, it fell from 34 on 2.11.08 to 28 on 06.04.09 :) (being almost the same in April 2009 as in April 2008).|
|Oct-22-10|| ||alexmagnus: 2010:
09.02: 37 (new max)
10.03: 38 (new max)
01.06: 39 (new max)
06.09: 40 (new max)
07.09: 41 (new max)
|Oct-22-10|| ||slomarko: <alexmagnus> and the conclusion is?|
|Oct-22-10|| ||alexmagnus: I just posted it for the sake of overview, you can draw your own conclusions.|
I personally stay with my no-inflation hypothesis. I already listed my arguments, but there is one more: shouldn't "only" inlftaion-driven growth be more or less constant, at least in such a short time frame? But we have sudden "jumps" and just as sudden "falls" (with the biggest fall being from 34 to 28).
I still think the main reason is the age shift - new players come, while old players are still good enough, so we have no "generation change". This is also why there was relative stability before 1986 (late 70s and early 80s we had what I call the "dying generation").
|Oct-22-10|| ||alexmagnus: Post-1986 my explanation is age shift+ dominator effect before Kasparov's peak.|
This also gives my future prognosis: the growth of number of 2700+ finishes as soon as the age shift is completed unless we get another dominator.
|Oct-22-10|| ||bartonlaos: <frogbert> Your player's page is flowing into more interesting areas, so I'll conclude this as succinctly as I can, hopefully to heal whatever damage I've inflicted in my arrogant way. |
<<If what you have admitted is true, then there should exist a correlation between the quicker site updates, and Carlsen's positive results. >
that would simply go to show that studies based on correlating events in order to establish causal relationships>
I was invited here to provide evidence of patterns. A positive correlation is an observed pattern. There's no need to go into causation. The pattern exists simply because Magnus is Magnus. When sincere members notice such patterns, don't take feelings to an extreme.
<the method is again flawed if your argument goes like this:> Your set-up for p% would be analyzed using an unpaired t-test, and not using total percentages. This is why you mistakenly thought the previous method was flawed. That's okay, not everyone cares so much about these things.
<the sooner the better> Right. Since returning to CG, I've noticed an increased hostility between members. Not just single players, but entire groups. That's not something I expected to find. I had tried to defend a player who was being aggressively attacked by many people. I thought they were sincere in their statement. Actually, they were part of some rival group looking for any excuse to stick a needle into your side. In the effort of fairness, I went out of my way to defend them by overinterpreting my experience. In doing so, I unknowingly added to their flame-fest. In the future I will definitely be more wary of whom I interact with on this website.
|Oct-23-10|| ||SwitchingQuylthulg: <frogbert> From what I remember classical inflation isn't likely to feature much in your hypothesis either, but for the sake of this discussion let's at least give it a clear definition:|
For any rating system S let p1 and p2 be two player pools of similar composition with time elapsing between their occurrence. If the relative strength of a player P compared to the rest of the pool has remained constant, classical inflation is taking place if and only if the rating of an average P has increased with time.
"Similar composition" here means that a) external factors such as aging have not significantly affected the overall strength of the pool; for example, if p1 consisted mainly of junior players, a rating increase would hardly indicate inflation of any kind, and b) the ratio of stronglings to middlings to weaklings has remained roughly constant; if, for example, the average rating within the pool has remained constant but the pool has seen a sudden influx of lowly-rated patzers, inflation is likely to have occurred. The pool(s) may or may not contain every rated player within system S.
|Oct-23-10|| ||alexmagnus: As for the rating boundary for live ratings... I suggest orienting at #1. As long as #1 is between 2775 and 2850, it remains at 2700. Then apply this prcedure:|
#1 falls below 2775 - lower the boundary to 2650
#1 rises above 2850 - rise the boundary to 2750.
Should #1 get over 2800 in the first scenario, get back two 2700 until he falls under 2775 again. Should he get under 2825 in the second scenario, get back to 2700 too until he rises over 2850 again. I hope the "general system" is clear for all rating changes.
Also an addendum for a case of Fischer-like dominating #1: the number of players on the list should be at least 20. If it is below 20, go to the lower boundary than indicated by the current position of the #1 (the boundaries are in 50-steps)
|Oct-23-10|| ||alexmagnus: Addendum to the addendum: maybe the minimum in such a case should be not even 20 but 10. As the number of player at the "lower boundary than indicated by the position of #1" may be huge.|
|Oct-23-10|| ||frogbert: sq, my short answer (for now, haven't got time to do any number crunching or to find examples) is this:|
1) the "coming of age" of the talented players born in the 80s (a clearly bigger group than the similar group of players born in the 70s and 60s btw)
2) the "resilience" of the players who emerged as "world elite" in the 90s (you know who i'm talking about - gelfand, anand, kramnik, topalov, svidler, ivanchuk, kamsky, adams, morozevich, leko, shirov ...)
3) a very moderate systemic inflation
of the above reasons, reason number 3 is the clearly least significant one, in my opinion. when i have time to study the numbers more closely i might find something that will make me have to revise this opinion, but currently this is what i think anyway.
i'll read your second post (and those of alexmagnus) later tonight. now i need to get back to the kids - i've been "managing" ("single-parenting"?) a 1-year-old and 7-year-old alone since thursday. i somehow agreed to letting my young wife run off to rome/italy together with a bunch of middle-aged men for 5 days ... ;o)
|Oct-23-10|| ||moronovich: <frogbert> and my wife goes off to Moss(Outside Oslo) tomorrow (for 4 days).. So may be we should share a couple of pints !?.;)|
And how about your son and his chess.
How is he/you doing !?
|Oct-23-10|| ||frogbert: sq, i collected some of those numbers that i think are relevant to explaining the observation you mentioned. these are trivial data to collect, so the question is whether they are relevant or not. personally, i think most of them are.|
it's a good day to do such an exercise - it's exactly 2,5 years (or 30 months) since <the first list was published on chess.liverating.org> on april 23rd 2008. (it was not the first top list of course, but let's start there anyway).
list 1: 2008042301 (apr 23rd 2008)
list 2: 2010102301 (oct 23rd 2010)
list 1: 27 entries
list 2: 39 entries
list 1: avg born year - 1977,5
list 2: avg born year - 1980,5
number of players within 100 points of #1
list 1: 27
list 2: 35
the "old" generation, here given the meaning "those born before 1980", had just recently become a "minority" on the top list on list1: they counted 13 out of 27 players. interestingly all of these 13 players can be found also on list 2, with one exception (polgar). let's have a look at the "golden oldies" on these two ("randomly selected") lists:
[name - list 1 rating - list 2 rating]
Anand - 2798,1 - 2802,1
Kramnik - 2788 - 2790,6
Moro - 2774,7 - 2700
Topalov - 2767 - 2777,1
Ivanchuk - 2750,7 - 2764,2
Shirov - 2741,4 - 2734,9
Leko - 2741 - 2716,5
Svidler - 2737,9 - 2724,6
Adams - 2736,7 - 2723,5
Kamsky - 2730,9 - 2725,9
Movsesian - 2725,9 - 2722
Gelfand - 2721,5 - - 2741
Polgar - 2709 - 2685,8
with a couple of notable exceptions (moro and leko), they have basically kept both their rating and their ranking order. it's interesting though that "the best" have gained a couple of points while "the 2nd best" (of their generation) have lost a couple points.
on list 1 basically 1/2 of the players were from the "old generation" - 13/27 or 48%. on list 2 there are still 13 players from the "old generation" (polgar exchanged with almasi), but now they only constitute 13/39 or 1/3 (33,3%) of the top list players.
before turning the attention to the "new faces", let's first notice that list 1 had just been expanded to include most of the great "class of 83": on the official fide list of january 2008 "only" <four players born in 1983 had a rating of 2700+> and equally interesting (maybe), among the top 10 on that fide list <only 2 players> did not belong to the "old generation". on list 2, only 4 players of the "old generation" are still present, the remaining 6 being born in the 80s or in 1990.
here are some live top list stats for a few golden years.
list 1 (2008):
born in 1983: 6
born in 1985: 3
born in 1987: 1
born in 1990: 2
list 2 (2010):
born in 1983: 7 (+1)
born in 1985: 3 (=)
born in 1987: 5 (+4)
born in 1990: 4 (+2)
in 2008 players born in 1983 turned 25, and basically all of those with potential to become "really good" broke 2700 before they were 24-25 (grischuk and ponomariov way earlier). the only new 1983-players on list 2 are jobava and dominguez (ni has dropped way out). of the 1985-players mamedyarov and alekseev remain, while bu dropped out and was "replaced" by navara (who's been in and out).
the talents of 1987 were only 21 years old in 2008, and the super-prodigy radjabov was the only representative back then. 2,5 years later he's been joined by wang yue (10th on list 2), nakamura (15th), wojtaszek (19th) and vitiugov (33rd), young radek of poland being the 2010 sensation in terms of massive rating gain among the best. so, the class of 1987 seems to bloom slightly earlier than that of 1983, and i assume most of those that will play a role in the chess world from that year has "shown themselves" before they were 22-23 - their current age.
the golden class of 1990 already had two representatives in "top 13" back in april 2008, despite the two being only 17 and 18 at the time. currently they are 4 above 2700, in the year they turn 20. maybe everyone with potential born in 1990 will have shown up before they are 21-22?
meanwhile, the old generation that initially learnt to play chess without all the fancy, modern tools seems to prove that talent, experience and hard work make it possible to keep up with the "computer generation" into the 40s - as long as you are serious and dedicated about it.
i doubt we will see players move the limits/records showcased by karjakin and carlsen [or kasparov by becoming "real" wc at 22] very much further. hence we're left with a window of roughly 20 years (20 to 40) where people theoretically can perform at elite level. how full "the pipe" will be will depend on several factors, and one important one is the economy of professional chess. if that one stays more or less the same, i'm tempted to think that the number of active 2700+ players might increase slower in the next 2,5 years.
|Oct-23-10|| ||frogbert: and unless someone goes "kasparov 1990" in terms of dominance, i expect the density among world-number-1-minus-100-points to continue to increase on average.|
|Oct-23-10|| ||frogbert: one can also speculate that being able to get close to the top faster and at a younger age might spur one to pursue a professional chess career, while a longer and more tedious road to a lucrative undertaking as a chess player would turn one off.|
in general, reducing this issue to a question about features of the rating system (or the amount of "inflation" specifically) quite clearly assumes way too many other relevant factors to remain constant (which they provably do not).
|Oct-24-10|| ||SwitchingQuylthulg: <frogbert> Thanks, good points. Can't find much fault with your reasoning; but while it's certainly true that "accusing" any features of the rating system of this growth is hasty, they (i.e. potential inflation, potential stretching or shrinking of the rating scale due to consistent over- or underperforming against lower rated players etc.) are still worth investigating, not just as possible contributors but to help understand the workings of the system itself.|
<and unless someone goes "kasparov 1990" in terms of dominance, i expect the density among world-number-1-minus-100-points to continue to increase on average.>
You're hoping a certain someone will, though, aren't you? :o)
1. Kasparov 2800
2. Karpov 2730
3. Gelfand/Ivanchuk 2680
Can't get much thinner than that... but there were more people within 100 points of #3 on that list than yesterday's list. That, probably, is where the "inflation! inflation!" people are coming from - it's not easy to tell just by looking whether yesterday's list is one where some 40 people have caught up with imaginary Ks, one where the Ks have been removed and everyone's been given an extra 100 points, or maybe a bit of both. (Yes, <of course> it would be extraordinary if there was an actual decrease in player density... but that's thinking, not looking.)
|Oct-24-10|| ||frogbert: <they (i.e. potential inflation, potential stretching or shrinking of the rating scale due to consistent over- or underperforming against lower rated players etc.) are still worth investigating, not just as possible contributors but to help understand the workings of the system itself.>|
certainly. and i've undertaken the only massive study i know of that uses <individual yearly rating changes> on a grand scale (essentially all players in the system) to create an estimate of systemic inflation since 1990. unlike the methodically flawed approach that (for some utterly strange reason) assumes the number of <strong players> to be exactly the same in 2010 as in 1990 in order to "prove" huge inflation, i've actually shown how people's ratings have changed.
i've also got data that correlate these rating changes with <age of players> and <activity of players>. as pointed out (directly) by bartonlaos and (indirectly) myself, one should be careful to assume causal effects only due to correlations - or at the minimum keep in mind that we're talking statistical relationships mainly. still, there are a number of broad tendencies waiting to be uncovered, potent to be capable of giving much better explanations to the increase in number of 2600+ or 2700+ players than the narrow-minded "inflation, inflation!" cries that are so common.
you might have read it here or elsewhere already, but based on one series of calculations made by yours truely (or technically, the script i wrote) i came up with a rough estimate of 30-35 points <systemic> inflation from 1990 to 2009.
of course, such an estimate doesn't imply that you can take the ratings of kasparov, karpov, gelfand and ivanchuk from 1990, add 35 points, and conclude that the following would be directly comparable to player ratings in the july 2009 list:
it does however result in much more "reasonable" ratings than the insane 120+ points of inflation since 1990 that sonas' "the rating of number 100" definition/approach leads to; consider
due to the inherent relativity of rating numbers (the close connection between a number and its pool) such exercises don't leave us with a tool for comparing players in 1990 with players in 2009, obviously. but i think it's encouraging that the methodically weak approach results in rather absurd adjusted numbers, while the much more methodically sound approach of mine resulted in something that feels kind of meaningful.
personally i'm quite sure that the two gentlemen gelfand and ivanchuk are more formidable opponents <on average> today (and during the last decade) than when they were 20-21. based on the reasoning in my previous posts, i'd rather think they weren't in a position to get close to their max levels before somewhere between 1994 and 2000, <after> having turned 25. [despite an occasional great event.] and as demonstrated by several (kasparov, anand, gelfand, ivanchuk etc. in recent times) i think your level of performance don't drop much from 30 to 40 if you stay active, work hard and adapt the playing frequency to your "system".
this might change with the generation that "became good" earlier in their lives (without necessarily having the huge drive of someone like kasparov). for instance, have we already witnessed leko reaching his career top around 25? will he retire before he's 35? it's interesting. the times they are a-changin'...
|Oct-24-10|| ||alexmagnus: <frog> What do you think of my idea that when the "age shift" gets completed, the increase of the number of 2700+ stops and we get a similar "stability" as in 1975-86 (unless another dominator comes up)?|
Actually when I speak about the "age shift" I mean two "age shifts" which followed quite closely. The first "age shift" happened with around the advent of Kasparov and is correlated with the "demise" of the "dying generation". As you know, I dubbed most top players during Karpov's championship the "dying generation" because of their ages (most of them 40+, some even 50+). Kasparov in his Candidates still had to do with this generation (Korchnoi in the semifinal, Smyslov in the final). I recently looked up if we ever else had such a "dying generation" phenomenon (based on chessmetrics ratings as something else is not there) - apparently the only time we had one is shortly before the outbreak of WW2.
Now, it's obviously that the players of the "dying generation", while being on top of the world, were far from their absolute top level. So when Kasparov and Co. came (and before him Karpov), they, young blood, easily overtook them - this caused increase in <top> ratings from ~2600-2700 to ~2700-2800, since 1987-1988 also driven by Kasparov's domination even higher. This "age shift" had also a similar effect as the current, second "age shift" - increase in number of strong players - the number of new young top players was somewhat bigger than the number of the "dying generation" members.
The second "age shift" happened because of computers. I cannot exactly date its beginning, but it's still not completed, as your numbers document. Old players still hold their own while more and more young players come. My theory: at some point the number of young players able to reach the necessary level will reach some "critical point" - more is impossible because there are not so many people talented enough. Then the number of strong players stabilizes. Once in a generation we'll get a new "injection of talents", but with each such "injection" the old generation will lose their ratings because of age (I think, the age-skills mapping will be not as now, when we have a "collision" of pre-computer and computer generation). So we get stability. In worst case maybe even another "dying generation" (the most talented players of computer generation reach top and stay there until their 50s, while the younger players will not have the necessary talent, with few Karpov-like exceptions).
I apologize if this post is somewhat chaotic and "illegible" - expressing my thoughts never was my strength :)
|Oct-24-10|| ||frogbert: <my idea that when the "age shift" gets completed, the increase of the number of 2700+ stops and we get a similar "stability" as in 1975-86>|
alexmagnus, i think there are interesting points about your theory, but i also think there are <more> factors that are relevant - for instance those pertaining to economy and number of events to participate in, and such.
it's not only an "age-shift" (linked to a technology-shift), there have also been a major change in recruiting, number of fide-rated tournaments (and hence, number of fide-rated players) and so on. while the big majority of the "new" players are neither new nor very talented (i.e. amateurs like myself and worse), it's quite obvious that there's been a notable increase in the number talented players too. it's not immediately obvious to me that such a development can't change direction if the market can't feed this new breed of professional players.
|Oct-24-10|| ||slomarko: <personally i'm quite sure that the two gentlemen gelfand and ivanchuk are more formidable opponents <on average> today (and during the last decade) than when they were 20-21.> |
are we talking about the same Ivanchuk who got crushed even by Hammer the other day!? come on frogbert, the version of Ivanchuk who beat Kasparov and Karpov to cruise to an easy win at Linares 1991 would have a field day with today's erratic Ivanchuk version.
|Oct-24-10|| ||alexmagnus: <come on frogbert, the version of Ivanchuk who beat Kasparov and Karpov to cruise to an easy win at Linares 1991 would have a field day with today's erratic Ivanchuk version.>|
Well, "erratic Ivanchuk"'s rating is changing often. When he is in bad shape, he often gets close to falling below 2700, while in good shape he reached his all-time high of 2787.
But the talk was about <1990>, not about <1991>. On the July 1991 list Ivanchuk reached his all-time high on my "domination list" (105 points ahead of #10, 4th place on the list), with a rating of <2735.>
< Earlier Kibitzing · PAGE 198 OF 242 ·