Chess discussions

[charlton66]

I watched one of Hikaru's videos about this and he was examining similar data. He then decided to pull up some of his own games and picked what he considered to be the 2 greatest games he ever played and ran the correlation. Both games came in around 80%. He was actually quite disappointed that he hadn't done better.

 

[Ekkie Thump]

I think the abovementioned video does a better job at explaining it, taking in account only Niemann's tournament performances and the very best streaks of Magnus/Fischer's career. And his average over 6 consecutive tournaments is significantly higher than the best ever streaks of similar length by Magnus (70%) and Fischer (72%). As @TheMagicFoolBus says, in the end it's a clear case of the Occam's Razor principle — unless we're talking about a potential legal case against Niemann, where you'll need to prove not the fact that he did it instead of eliminating any reasonable doubt in the probability of it happening without cheating.

I'll definitely watch it later. I'm just acutely aware that an enormous number of eyes are staring at Niemann's games right now with the single endeavour of finding out anything and everything remotely suspicious. This leads to a greater degree of confirmation bias than usual. We have to make sure that whoever's coming up with the results are competent and disinterested - not just bandwaggoning Texas sharpshooters. I guess if multiple different sources are coming to the same results then that makes a difference.

Like if this Johannes fella also conducted the same analysis on Niemann and a bunch of other GMs on the rise then published the parameters and results so anyone with ChessBase could have a go then I'd be a lot more comfortable. I guess that something like this must be in the process of being done so it won't take that long to find out.

I agree the blokes probably cheating, I'm just not (yet) convinced he's cheating.

 

[JPRouve]

I'll definitely watch it later. I'm just acutely aware that an enormous number of eyes are staring at Niemann's games right now with the single endeavour of finding out anything and everything remotely suspicious. This leads to a greater degree of confirmation bias than usual. We have to make sure that whoever's coming up with the results are competent and disinterested - not just bandwaggoning Texas sharpshooters. I guess if multiple different sources are coming to the same results then that makes a difference.

Like if this Johannes fella also conducted the same analysis on Niemann and a bunch of other GMs on the rise then published the parameters and results so anyone with ChessBase could have a go then I'd be a lot more comfortable. I guess that something like this must be in the process of being done so it won't take that long to find out.

I agree the blokes probably cheating, I'm just not (yet) convinced he's cheating.

It has been done already for some like Erigaisi who is himself on an incredible rise.

Tweet

— Twitter API (@user) date

At the moment no one is close to Niemann's stats. Give it a week or two and we will have more exhaustive set of data but currently Niemann is better than all super GMs and also Bobby Fisher.

 

[Abraxas]

Can someone explain to me how one can cheat only occasionally on this level?

If I boost my ELO in chess/go by using an engine, I'll get rolled over the second I stop using it. Its a bit different if you are already able to compete at the highest level and than use engines in some games, but that doesn't seem to be the situation, that Niemann is in. Its at least insinuated, that his fast rise was helped by engines. Even in the Sinquefield Cup, Hans was competitive in most games. So in short: wouldn't Hans be forced to cheat always if he plays these super GMs or risk getting exposed?

Not really because Hans is a young and emotional player, he can get away with bouts of inconsistency because that can be typical. In fact, in top level chess unless your name is Magnus Carlsen it is more or less hard to predict who is going to perform in a tournament. So it doesn't raise any eyebrows that he gets to the top table and gets crushed sometimes, the point is perhaps that he used the engine to help his rise to that point quickly.

Also I don't think Hans is a 2300 player using an engine to play like a 2700 player. Hans is certainly a good Grandmaster, we know that because it's obvious he is not cheating at every moment or even in every game and he can play well. Which means that cheating can actually be a lot more subtle than if you or I were to attempt it. We may say that he's not even subtle, but if you consider that the only reason this is an issue is because Magnus threw a paddy, and that still many are not convinced, it was subtle enough.

 

[do.ob]

Not really because Hans is a young and emotional player, he can get away with bouts of inconsistency because that can be typical. In fact, in top level chess unless your name is Magnus Carlsen it is more or less hard to predict who is going to perform in a tournament. So it doesn't raise any eyebrows that he gets to the top table and gets crushed sometimes, the point is perhaps that he used the engine to help his rise to that point quickly.

Also I don't think Hans is a 2300 player using an engine to play like a 2700 player. Hans is certainly a good Grandmaster, we know that because it's obvious he is not cheating at every moment or even in every game and he can play well. Which means that cheating can actually be a lot more subtle than if you or I were to attempt it. We may say that he's not even subtle, but if you consider that the only reason this is an issue is because Magnus threw a paddy, and that still many are not convinced, it was subtle enough.

Chess.com caught him several times and other players clearly smelled something fishy, hence raising concerns about him with the Sinquefield Cup org. This wasn't an issue, because the authorities didn't want to make it one - not because Niemann was exceedingly subtle. What people are doing now, trying to connect the statistical dots, should have been done as the tournaments were played, it shouldn't have taken a heavy handed prompt from Carlsen. The same goes for chess.com's opinions. We can only hope that as a consequence of this drama cheating issues are tackled more proactively.

 

[MDFC Manager]

I'll definitely watch it later. I'm just acutely aware that an enormous number of eyes are staring at Niemann's games right now with the single endeavour of finding out anything and everything remotely suspicious. This leads to a greater degree of confirmation bias than usual. We have to make sure that whoever's coming up with the results are competent and disinterested - not just bandwaggoning Texas sharpshooters. I guess if multiple different sources are coming to the same results then that makes a difference.

Like if this Johannes fella also conducted the same analysis on Niemann and a bunch of other GMs on the rise then published the parameters and results so anyone with ChessBase could have a go then I'd be a lot more comfortable. I guess that something like this must be in the process of being done so it won't take that long to find out.

I agree the blokes probably cheating, I'm just not (yet) convinced he's cheating.

If he's admitted to cheating, plus been actually caught cheating, that's enough surely?

 

[HookedOnAPhelan]

He even looks like a cheating cnut.

 

[harms]

I do think that 100% in longer games is a lot less defensible than those just out of known theory or what have you. So again, suspicious.

If a game mostly consists of known theory, the engine doesn’t calculate it. Hikaru tried it in a few games where the first 15-17 moves were theory and the engine said that those games didn’t have enough moves (meaning original ones) to analyze them.

 

[Abraxas]

Chess.com caught him several times and other players clearly smelled something fishy, hence raising concerns about him with the Sinquefield Cup org. This wasn't an issue, because the authorities didn't want to make it one - not because Niemann was exceedingly subtle. What people are doing now, trying to connect the statistical dots, should have been done as the tournaments were played, it shouldn't have taken a heavy handed prompt from Carlsen. The same goes for chess.com's opinions. We can only hope that as a consequence of this drama cheating issues are tackled more proactively.

I'm not talking about online. Chess.com would lead us to believe their algorithms are brilliant so therefore it should be catching subtle usage anyway.

But over the board he's done enough that not everyone agrees he has cheated OTB, and certainly not many want to come out and declare a definitive opinion.

 

[JPRouve]

If a game mostly consists of known theory, the engine doesn’t calculate it. Hikaru tried it in a few games where the first 15-17 moves were theory and the engine says that those games don’t have enough moves (meaning original ones) to analyze them.

That part made the analysis more credible because he even took a longer game that was almost entirely theory and it said that there too few moves. I interpreted it as the analysis is focused on new positions or not theorized positions which makes high percentage accuracies even more impressive because it would be based on intuition and the ability to read and analyze a position live.

 

[Abraxas]

I would like to see the hypothesis that Hans playing weaker players increases the probability of 90-100% results tested.

My intuition is that this may have some impact on results because mistakes that are more elementary are more likely to be clear to both the engine and a top player. However, it also has to be said that cleanly converting precisely according to the engine is not trivial. Humans make practical decisions and engines do not, they simply calculate objectively best lines according to their logic.

So it strikes me that it would still be very hard to score 90-100% correlation. But this would be really easily tested as strong players play weaker titled players in Olympiads and open tournaments.

 

[Jotun]

I would like to see the hypothesis that Hans playing weaker players increases the probability of 90-100% results tested.

My intuition is that this may have some impact on results because mistakes that are more elementary are more likely to be clear to both the engine and a top player. However, it also has to be said that cleanly converting precisely according to the engine is not trivial. Humans make practical decisions and engines do not, they simply calculate objectively best lines according to their logic.

So it strikes me that it would still be very hard to score 90-100% correlation. But this would be really easily tested as strong players play weaker titled players in Olympiads and open tournaments.

I don't think there are many elementary mistakes above 2000. People are acting like he was playing against casuals, that have blunders like losing queens or major pieces. At this level, blunders are losing pawns.

 

[JPRouve]

I don't think there are many elementary mistakes above 2000. People are acting like he was playing against casuals, that have blunders like losing queens or major pieces. At this level, blunders are losing pawns.

And it's not even the point. Here the "competitor" is the engine itself, since the point is about playing the top engine move repeatedly. Also here the 10 100% correlation games, it's not against weak players.
https://lichess.org/study/ffYRNE1u

 

[Abraxas]

I don't think there are many elementary mistakes above 2000. People are acting like he was playing against casuals, that have blunders like losing queens or major pieces. At this level, blunders are losing pawns.

It's all relative. 2000s make loads of mistakes that are elementary for players at 2700. I reached 1800 in my heydey and I blundered constantly. Not usually 2 move tactics, but trivial strategic oversights and eventually end up in bad positions where a further blunder losing material is inevitable.

If we go with the hypothesis that Hans was not cheating but had substantially improved during Covid (see Prag, Erigaisi, Abdusattarov, Keymer and many others) then he ought to have been substantially stronger than his rating suggested, meaning some of the opponents in Yosha's sample were much weaker players. I believe that's where this argument arises.

I'm not even strongly into it to be honest, I think it's still hard to play with the consistency of the comp. But it's a legit line of enquiry. A lot of precision is missing from some of the statistical analysis.

 

[TheMagicFoolBus]

Thanks for that. Yeah - to be clear I think on the balance of probabilities Niemann is likely a cheater. He's admitted cheating, he's been called out further by Chess.com and there are now these weird statistical anomalies. It's just I'm naturally cautious about using data to reach stronger conclusions than should be. Science is littered with observations which appear at first glance to be statistically significant but turn out not to be once more rigour has been applied.

To your points. If the net was drawn as widely as possible for Magnus then that's great - it must have picked up as many 90%+ games as is possible so yeah, that lessens my concerns.

I do think that 100% in longer games is a lot less defensible than those just out of known theory or what have you. So again, suspicious. I just think it's probably a lot easier to play 100% against a 2600 than against a 2750. Obviously you'd have to play way above 2600 to do it, but while unlikely it's not without the realm of possibility that Niemann can play to those standards on occasion. I guess I'd just prefer it if analyses were done on similar up and comers with similar opponents. I guess this has already been done or is being done as I type.

A minor quibble with your first point. After a quick eyeball of Deleng's histogram I'd suggest his dataset contains far more than 278 games. At a guess I'd say 380+ games went into producing that. Not sure if it's the same dataset or what though. Edit: This seems to be the original Niemann dataset. I computed 407 games listed (but I might be misunderstanding/miscalculating).

No worries mate and apologies if I've misrepresented anything. I think you're right to be skeptical (and I'm sure my old Russian PhD advisor would have kittens if he saw me throwing around figures like this willy-nilly), but I personally view this as a case of Occam's Razor as I mentioned earlier. For me the null hypothesis strains all credulity, even if that isn't purely on a statistical basis.

 

[Kazi]

I just want to know how he is doing it and then I can get back to my normal life

 

[JPRouve]

No worries mate and apologies if I've misrepresented anything. I think you're right to be skeptical (and I'm sure my old Russian PhD advisor would have kittens if he saw me throwing around figures like this willy-nilly), but I personally view this as a case of Occam's Razor as I mentioned earlier. For me the null hypothesis strains all credulity, even if that isn't purely on a statistical basis.

Do your worse and give us a breakdown of this one.

 

[One Night Only]

He looks like a baddy tbh.

 

[Nani Nana]

I just want to know how he is doing it and then I can get back to my normal life

Anal beads

 

[do.ob]

I'm not talking about online. Chess.com would lead us to believe their algorithms are brilliant so therefore it should be catching subtle usage anyway.

But over the board he's done enough that not everyone agrees he has cheated OTB, and certainly not many want to come out and declare a definitive opinion.

I don't see what's subtle about playing the top engine move 12(?) times in a row, like Caruana suggested. Or having a phenomenal rise in elo, or drawing attention to yourself, by having a 5-ish tournament streak that beats Fischer's peak, like this engine correlation score suggests. There appear to be quite a few indicators about the guy, that just jump into your face. It's just the nature of high level chess, that it's impossible to find direct proof for a strong GM cheating, unless you catch them red-handed. That's why people water down their statements, because they don't have anything concrete to defend themselves in case of legal action from Niemann himself or FIDE's ethics comission.
To me it looks like the system is not set up to catch cheaters via statistical analysis - rather than Niemann being clever - which is why we're stuck with Twitter analysts and subjective opinions from top players, rather than genuine expert opinions, which we could hope to treat as facts.

It's different for chess.com, they have the right to control access to their platform, they don't have to defend their decision to close an account in court, so they can just nuke an account, if they are personally convinced.

 

[VanDeBank]

I think if he cheated at smaller tournaments, he should have his ass handed to him at a higher level once he's out of book.

It depends on a number of factors that aren't accounted for in the tweet. Not saying it's not suspicious but statistical analysis is very easy to feck up and my questions would be as follows:

1. Did Carlsen and Niemann play a similar number of games over the sample period? Rule 101 of statistical analysis is to express this type of comparison in terms of a rate - using absolute numbers tells us very little on its own. If Carlsen played 100 games and Niemann played 1,000 then Carlsen's 100% rate would be twice that of Niemann's despite the headline number being 2 vs 10.

2. Were their opponents of a similar stature? My intuition would be that it is easier to play closer to perfect when the opponent plays badly. My thesis is that your opportunities would be more plentiful, often more obvious and the path forward often simpler to infer/deduce. My suspicion is that Niemann's pool of games probably contains a greater proportion of open tournaments than Carlsen's - who likely plays more frequently in invitationals against fellow SuperGM's. I'd contend that because of this Hans likely has a greater number and proportion of games involving inferior opponents against whom it was easier to "play well". For me this single difference might fundamentally skew the two datasets and for this reason I'd suggest Magnus might be a less than ideal point of comparison

3. What were the parameters of the analysis and are we sure they were the same for both? As far as I can work out ChessBase uses a random number of cloud sourced chess engines in order to compute the games and does so according to some number of predefined user constraints. I *think* that in order for a move to count as 'engine correlated' it must be the top suggested line on any one of the 15-25 random engines of differing abilities currently providing the analysis. This is already quite a broad net. Given that the engines ChessBase uses constantly change in real time it's clear that no two analyses can ever be truly identical even if they use identical inputs on identical data (though it would probably be quite close). I've also heard tell (not sure) that the definition of what counts as 'engine correlated' might be expanded by the user to include the top 3 lines - this would obviously have the effect of greatly expanding the number of moves counting as such. At any rate, what is certain is that the tool's sensitivity can be manipulated by the user in multiple other ways prior to its run (engine depth, time constraint, use of opening book etc). Given that the analysis of Niemann was done by a different person at a different time with unclear user defined parameters it's difficult to know to what extent that analysis can be compared to the one done here on Carlsen.

In short, far more rigour needs to be employed in the production and comparison of the analyses for us to be able to derive meaning from them.

@no 2, in my personal experience, you take a safety first approach playing a weaker player and this could be purposefully suboptimal to reduce variance.

This was a good read. Thanks.

 

[JPRouve]

I think if he cheated at smaller tournaments, he should have his ass handed to him at a higher level once he's out of book.

That point isn't true, in fact his coach is a known cheat and a solid high level player. Sebastien Feller has been caught cheating and he is currently a 2540 rated player. Other cheaters that have been caught are genuine +2600 rated players. None of these would or have had their ass handed to themselves because they are actually good but Chess is incredibly competitive and the difference between winning, drawing and losing is small, it's even smaller at the highest level.

And that's the entire issue with this. While Niemann has cheated in the past, outside of catching him in the act it's impossible to be 100% sure about which games he cheated on unless he tells you.

 

[Abraxas]

I don't see what's subtle about playing the top engine move 12(?) times in a row, like Caruana suggested. Or having a phenomenal rise in elo, or drawing attention to yourself, by having a 5-ish tournament streak that beats Fischer's peak, like this engine correlation score suggests. There appear to be quite a few indicators about the guy, that just jump into your face. It's just the nature of high level chess, that it's impossible to find direct proof for a strong GM cheating, unless you catch them red-handed. That's why people water down their statements, because they don't have anything concrete to defend themselves in case of legal action from Niemann himself or FIDE's ethics comission.
To me it looks like the system is not set up to catch cheaters via statistical analysis - rather than Niemann being clever - which is why we're stuck with Twitter analysts and subjective opinions from top players, rather than genuine expert opinions, which we could hope to treat as facts.

It's different for chess.com, they have the right to control access to their platform, they don't have to defend their decision to close an account in court, so they can just nuke an account, if they are personally convinced.

To be honest I don't really know what the importance of this debate is, but anyway it was subtle enough that there isn't total agreement and subtle enough to get him invites to big money tournaments and earn decent chunks of money and subtle enough that he's gone so far as to bring the whole game into disrepute. So that's damage done and job done for a cheater, albeit I am sure he regrets it now. Whether that's because of poor detection, cautiousness in accusing others of cheating, it doesn't really matter because the cheater operates within the climate that's present at the time.

The fact is it took an unprecedented decision from Magnus to have Hans scrutinised for his OTB play. Players have been griping about other players for absolutely ages, there's nothing new about suspicion - it has all been buried and ignored for years. The sole differentiation is Magnus, and yes that is probably because detection is poor and cheating is not taken seriously enough, nor is there joined up thinking within chess, which is precisely the reason Magnus has done it judging by his statements.

 

[TheMagicFoolBus]

Do your worse and give us a breakdown of this one.

Sorry but I think you are responding to the wrong person as I believe we agree on this?

 

[Kasper]

Shout-out to the guy that managed to keep up for 45 moves with Don Stockfish Niemann in one of these 100% games, proper legend haha.

Meanwhile, Hans is confirmed to play the US championship so I hope he pulls out some proper wild troll moves to throw people off because they'll think they're playing Alpha Zero. Firouzja was shellshocked in the singfield cup after the game against him.

 

[Zarlak]

Shout-out to the guy that managed to keep up for 45 moves with Don Stockfish Niemann in one of these 100% games, proper legend haha.

Meanwhile, Hans is confirmed to play the US championship so I hope he pulls out some proper wild troll moves to throw people off because they'll think they're playing Alpha Zero. Firouzja was shellshocked in the singfield cup after the game against him.

Inb4 opponent was using Stockfish too.

 

[do.ob]

To be honest I don't really know what the importance of this debate is, but anyway it was subtle enough that there isn't total agreement and subtle enough to get him invites to big money tournaments and earn decent chunks of money and subtle enough that he's gone so far as to bring the whole game into disrepute. So that's damage done and job done for a cheater, albeit I am sure he regrets it now. Whether that's because of poor detection, cautiousness in accusing others of cheating, it doesn't really matter because the cheater operates within the climate that's present at the time.

The fact is it took an unprecedented decision from Magnus to have Hans scrutinised for his OTB play. Players have been griping about other players for absolutely ages, there's nothing new about suspicion - it has all been buried and ignored for years. The sole differentiation is Magnus, and yes that is probably because detection is poor and cheating is not taken seriously enough, nor is there joined up thinking within chess, which is precisely the reason Magnus has done it judging by his statements.

To me it's an important distinction, because I think it's a systemic issue, caused by inaction from FIDE, but also chess.com, who didn't report Niemann when they banned him.
So I think the lesson from this has to be much more investment in anti cheating measures.
As you've alluded to in your second paragraph I also think Niemann not being particular subtle is an important factor when assessing Carlsen's behavior, which could look entirely different in a vacuum.

 

[berbatrick]

Meanwhile, Hans is confirmed to play the US championship so I hope he pulls out some proper wild troll moves to throw people off because they'll think they're playing Alpha Zero. Firouzja was shellshocked in the singfield cup after the game against him.

The game ... which he drew after Hans gave up his advantage and left himself open to a winning attack that Alireza couldn't find.
I'm really struggling to see how he used any engine help in that tournament.

 

[do.ob]

The game ... which he drew after Hans gave up his advantage and left himself open to a winning attack that Alireza couldn't find.
I'm really struggling to see how he used any engine help in that tournament.

Who knows for sure what was going on at that tournament, could be that he never cheated there could be that he did and they had to stop (limit themselves to 1 or 2 moves per game) after the drama started.
If he has a mental edge over other players due to his cheating that's another reason why he shouldnt be invited to tournaments.

 

[sullydnl]

Tweet

— Twitter API (@user) date

 

[Kasper]

The game ... which he drew after Hans gave up his advantage and left himself open to a winning attack that Alireza couldn't find.
I'm really struggling to see how he used any engine help in that tournament.

Oh, I don't think he did. But I'm pretty sure he's considering playing deliberately obscure moves to throw his opponents off now that everyone knows about the allegations. Firouzja was pretty rattled by Qg3 so that clearly worked.

 

[JPRouve]

Oh, I don't think he did. But I'm pretty sure he's considering playing deliberately obscure moves to throw his opponents off now that everyone knows about the allegations. Firouzja was pretty rattled by Qg3 so that clearly worked.

It's a bit of a stretch to say that Firouzja was rattled or shocked. He thought that it was a senseless sacrifice because there was no obvious compensation.

 

[Kasper]

It's a bit of a stretch to say that Firouzja was rattled or shocked. He thought that it was a senseless sacrifice because there was no obvious compensation.

Then why didn't he take?

 

[JPRouve]

Then why didn't he take?

I believe that it's Caruana that explained that in an unrelated context. When top GMs see something that look strange they have the tendency to be conservative and in this case it made particular sense since Firouzja was playing black, his first goal was a draw not to get into a sharp position.

 

[Kasper]

I believe that it's Caruana that explained that in an unrelated context. When top GMs see something that look strange they have the tendency to be conservative and in this case it made particular sense since Firouzja was playing black, his first goal was a draw not to get into a sharp position.

Oh yeah, that does happen and Hans explained this as the very much reason why he made that move later. But the psychological effect of the cheating rumour definitely had an impact as well.

 

[Ødegaard]

Again, i don't think he cheated in the tournament. But i do think he gets an unfair psycological advantage over his opponents by being a known cheater.

I honestly think any professional who cheats should be permanently banned if they do it after age 25, and banned until aged 25 if they do it before.

 

[do.ob]

Tweet

— Twitter API (@user) date

popcorn.gif

 

[sullydnl]

Tweet

— Twitter API (@user) date

 

[Fortitude]

Isn’t that him done? How can he possibly recover from that? His reputation is that of a full-on cheat.

 

[TheMagicFoolBus]

Niemann is toast. No fecking chance he comes back from this.

Hope all those who treated Magnus unfairly regret it. He did absolutely nothing wrong and is completely vindicated. All that remains is figuring out how Niemann cheated OTB but in my mind there's zero doubt he did so.