Life in 19x19. Go, Weiqi, Baduk... Thats the life.
https://lifein19x19.com/
launched the application
My saviour!Uberdude wrote:launched the application
means double clicked leelaz.exe? You should double click Lizzie.jar and, assuming you have .jar files associated correctly, that should open the lizzie interface. If not open a command prompt in that folder and run "java -jar Lizzie.jar".
Unfortunately, it depends. I would still happily follow LZ's judgment in the early game even though it was mistaken about the size of the komi; maybe it thinks that the winrate is 46% while it is really 60% or something, but it'll still make reasonable moves. On the other hand, at the very end of the game, if Black is ahead by 3 points on the board, say, it will think that Black's winrate is almost 0% when it really should be almost 100%. So I think its endgame evaluations in a close game (as will happen in most pro games) will be pretty useless.John Fairbairn wrote:Second, another question. I have read that the AI programs don't work happily with different komis. But presumably this has been discussed. I'd love to know what the general feeling is as to how much less reliable LZ is if the game komi is zero, or more specifically if LZ starts with a winrate of say 46% for Black on an empty board in a no-komi game (i.e. it still believes komi of 7.5 applies), what is the general LZ-assisted human view as to the real winrate knowing that komi is actually 0 (and likewise for various handicaps)?
There are other reasons to stress efficiency, I think.John Fairbairn wrote:First a comment. I have mentioned several times before that pros usually don't count the way we do. They can look at the board and see inefficiencies and simply count those (there aren't so many in pro games). Generally the side with the most inefficiencies is behind, but obviously some inefficiencies are worse than others and pros seem to have enough skill to assess how much each one is worth (I expect, though, some are significantly more skilled at this than others). Because they have this feel for the game, they stress efficiency of plays an awful lot.
There are tricks, such as the saying that ponnuki is worth 30 points. Even the idea that one handicap stone is worth 10 points is sort of a trick. But we now know, because of komi, that a handicap stone is worth around 14 points (to a good player). That's a 40% difference.I've understood this, and have even been able on occasion to simulate their behaviour, but mostly I have treated it as a little bit of a party trick by them.
It takes experience to develop judgement. Modern bots enjoy the experience of millions of games. No human can match that.However, my first forays with Lizzie/LZ have astonished me because it spots inefficiencies very early in the game and marks them down heavily. For example, there was a joseki where a connection was needed and two were available. The pro played the solid connection but LZ preferred a hanging connection. It hardly seemed to matter because there was no immediate danger, no shortage of liberties, nothing special at all - purely a case of long-term efficiency. But LZ adjudged the pro's solid move a whopping 6 percentage points worse. That pattern seems to emerge elsewhere and so I now understand even better why pros care about efficiency of plays, but at the same time it seems they have not entirely mastered all the elements of good shape.
In the 1970s I saw some Nihon Kiin statistics on 2800 pro games, 1400 played with 4.5 komi, 1400 played with 5.5 komi. It was partly on that basis that I predicted that Japanese komi would be 6.5 by the year 2,000. (Almost.Second, another question. I have read that the AI programs don't work happily with different komis. But presumably this has been discussed. I'd love to know what the general feeling is as to how much less reliable LZ is if the game komi is zero, or more specifically if LZ starts with a winrate of say 46% for Black on an empty board in a no-komi game (i.e. it still believes komi of 7.5 applies), what is the general LZ-assisted human view as to the real winrate knowing that komi is actually 0 (and likewise for various handicaps)?
) For both sets of statistics the median result was between 6 and 7 points on the board for Black. Also, IIRC, if you applied different komis around 6.5 to the data, a one point difference in komi meant a difference in win rates of 2% plus. Leela Zero uses area scoring, where minimum score differences are nearly always at least 2 points. So a minimum score difference translates to a winrate difference of around 4-5%. So the pro's mistake (if Leela Zero is right) cost maybe 2-3 pts. on the board.
2. There is an experimental version of LeelaZero that works for 0 and other komis (but it's not as easy to install). It basically does a trick of switching black and white and going for the middle and hoping the magic of the neural networks is basically linear, and it seems to work fairly well (though not being self-play trained in these conditions means it'll be less accurate). With network #153 (LZ's opinion does slowly evolve with new versions) it thinks black starts off at 61.5% on the empty board with no komi (cf 46.5 with 7.5 komi). I did a little analysis of Shusaku's famous ear-reddening move with it here.John Fairbairn wrote:Second, another question. I have read that the AI programs don't work happily with different komis. But presumably this has been discussed. I'd love to know what the general feeling is as to how much less reliable LZ is if the game komi is zero, or more specifically if LZ starts with a winrate of say 46% for Black on an empty board in a no-komi game (i.e. it still believes komi of 7.5 applies), what is the general LZ-assisted human view as to the real winrate knowing that komi is actually 0 (and likewise for various handicaps)?
Code: Select all
Black starts, white 7.5 komi ("even game"): 46.7
Black starts, no komi 65.5
Black starts with 2 stones, white 7.5 komi 80.7
Black 2 stones, no komi 88.8
Black 3 stones, 7.5 komi 92.6
Black 3 stones, no komi 93.2
Black 4 stones, 7.5 komi 96.5
Black 4 stones, no komi 88.4 (yes, really, don't trust interpolation/extrapolation too much when you adventure to new lands)
Well, I rarely count points in games, and usually use the "sum of how bad I think my mistakes were" vs "sum of how bad I think opponent's mistakes were" as the basis for if I think I'm leading or not. I did actually count a few times in my recent title match game: in the middlegame my count suggested playing to defend my 3-3 could be ok but there was just too much play left in other areas of the board I wasn't confident to play a conservative gote move, and then things got out of hand and I never got the chance, and in the endgame I counted it was super close so did some reading instead of being lazy and playing (just!) unnecessary defence. I have noticed that in his commentaries Myungwan Kim counts territory a lot (though often does the "white needs to make at least 15 in this hard to count area to be even" approach) whereas Michael Redmond rarely does.John Fairbairn wrote:First a comment. I have mentioned several times before that pros usually don't count the way we do. They can look at the board and see inefficiencies and simply count those (there aren't so many in pro games). Generally the side with the most inefficiencies is behind, but obviously some inefficiencies are worse than others and pros seem to have enough skill to assess how much each one is worth (I expect, though, some are significantly more skilled at this than others). Because they have this feel for the game, they stress efficiency of plays an awful lot.
I think far too much judgement by pros of josekis (or at least what I've gleaned of it) is based on reference/tewari/comparison (where such comparisons can accumulate asymmetric errors) to other results they think (possibly mistakenly) are ok, instead of trying to make more absolute judgements based on where the stones end up. "Well we played these moves which are probably okay and it started even so it ended even" isn't great. Robert Jasiek's territory/influence stone counting approach is a nice idea in that direction, but just not good enough to be useful. Basically you need a massive function with gazillions of parameters finely tuned to judge a multitude of facets of a position. Hang on a minute, I just described a neural network!John Fairbairn wrote: I've understood this, and have even been able on occasion to simulate their behaviour, but mostly I have treated it as a little bit of a party trick by them. However, my first forays with Lizzie/LZ have astonished me because it spots inefficiencies very early in the game and marks them down heavily. For example, there was a joseki where a connection was needed and two were available. The pro played the solid connection but LZ preferred a hanging connection. It hardly seemed to matter because there was no immediate danger, no shortage of liberties, nothing special at all - purely a case of long-term efficiency. But LZ adjudged the pro's solid move a whopping 6 percentage points worse. That pattern seems to emerge elsewhere and so I now understand even better why pros care about efficiency of plays, but at the same time it seems they have not entirely mastered all the elements of good shape.
. I was recently reviewing the 5th Kisei title match of Shuko vs Otake with Go world commentary and half the opening moves were 8-12% point mistakes according to Elf (lots of splitting or extending on side moves, even before bots that had gone out of fashion as not active/efficient enough). LZ #157 was much more tame, just rating them -1 to -3%. Here's a graph from LZ (the horizontal dashed lines are at 25/50/75%).
) Btw, white's 40 was the biggest mistake of the opening at -9%, can you find the better move? It's totally logical and a beautiful tesuji that exploits black's timing mistake. LZ is good at this sort of sharpness in fighting. That big spike at move 110 is for the f11/g11 exchange that I couldn't understand and neither can LZ, saying it was bad. I'd really like to know what the players were thinking: did they see something LZ didn't which when shown LZ agrees was important (well done them), or did LZ not see it but when shown it thinks it's bad and sticks to its opinion, or did LZ see and dismiss it as bad? Also some of the big zigzags later I think are due to foibles in LZ's analysis, for example it wants to play the profitable sente exchanges from t13 for white quite early, so when white does something else it sometimes doesn't see (at 10k playouts) that white will be able to play them later too so thinks the other move is worse when really it's just a different order of chunks. Being a close game big swings could just be 1 point mistakes that change the winner. Also it wastes playouts throughout the game wanting to play black's a13 sente move, which humans can save for a ko threat with little thought and then stop wasting brainpower on. And generally I think the fighting might be too complicated for LZ on moderate playouts that I wouldn't confidently say they must be making big mistakes here. With more time I can try to tease out the truth, but really a stronger player than myself is needed for a reliable interpretation (Elf can help).This is definitely something to keep in mind. I did a similar exercise running old Go World commented games through LZ - there was one where a sequence was played that LZ heavily disliked, but that was because it didn't see that there was a ko left for later. Once the move that set up the ko was played, the winrate shot back up. So one has to keep in mind that the networks have these kinds of holes in their evaluations, and mentally correct for it if one manages to notice the problem.Uberdude wrote:Something to note is LZ didn't anticipate Otake's good move 29 beforehand so its judgements preceding that are less trustworthy, but once it gets to that move it does find it after about 30,000 playouts
I think that one reason that modern engines don't like moyos as much is that they are really good at reducing and invading them, so they don't think they are as valuable. I think in some of these cases a top human would also be able to invade and live/run if he really had to, but he'd really rather not stake the game on it, whereas LZ etc. are much more confident that they'll find a way.John Fairbairn wrote:LZ didn't like moyo-making moves. Humans did.
My interpretation of the last two points, at east in this game, is that LZ preferred solidity. Humans preferred flexibility. That surprised me. I'd have thought the AI program would prefer flexibility (they "see" farther") and humans tend to go for solidity as a way of reducing uncertainty (and especially so in no-komi games).
I'm curious whether whether it liked the move after it was played (what happened to the winrate).There was a tesuji that was universally praised by humans that LZ didn't even consider even when I let it run a longish time.
I put your "un" in brackets as I think it was a typo. Indeed if Black is winning by 3 points early in the game, he's ahead but White still has a chance, while if Black is winning by 3 points in the endgame, White might as well resign.At move 66, when commentators though the game was no close, the winrate in favour of White had shot up from 54% to over 80%, with no obvious horrendous mistakes - just a slow accumulation of increments. This of course assumes 7.5 komi. I'm guessing that what this means, at least in part, is that as the game progresses so does the [un]certainty and so the program can be ever more confident about its winrate. In other words, it does not necessarily mean Black's play deteriorated. Another 66 moves further on, again with no obvious blunders according to humans, the winrate had increased to 87% (and after a further tranche of 66 moves it was 99.8%).
Winrate is how probable LZ thinks it is that Black would win if it were allowed to take over the game and play both sides from here to the end. So certainly if move A has a higher winrate than move B you can infer that LZ thinks that move A is better, without having to worry about exactly what each number means.And, for me at least, it remains puzzling what winrate really means, but I infer it matters more in relation to individual moves rather than over a whole game.
I haven't got round to advanced thinking like that yetI'm curious whether whether it liked the move after it was played (what happened to the winrate).
It wasn't a typo though of course I may have made a mistake in other ways. My thinking was that as the board fills up and there are fewer moves to the end of the game (i.e. there is more information), it is possible to make more moves that are reliable (even if not always the best).put your "un" in brackets as I think it was a typo.
Yes, I understand that. But that is just when comparing A, B, C at the same move. What I was postulating was that, nearer the end of the game, the winrate for each of A, B and C is likely to be higher because of the "certainty" I already alluded to. That is, winrate at move 100 implies something a bit different from winrate at move 50. I suppose what I am really referring to is the winrate graph and am implying that its trend is not really telling us anything. The only parts that seem hugely significant are when the direction of the line changes drastically.Winrate is how probable LZ thinks it is that Black would win if it were allowed to take over the game and play both sides from here to the end. So certainly if move A has a higher winrate than move B you can infer that LZ thinks that move A is better, without having to worry about exactly what each number means.
My take, largely based upon AlphaGo, which may not hold for other bots, is that AlphaGo, and especially AlphaGo Zero, when playing itself, was extremely flexible. However, when playing humans, it often went for solidity. My guess is that when it felt itself sufficiently ahead, even though humans might not realize that, it went for solid play.dfan wrote:I think that one reason that modern engines don't like moyos as much is that they are really good at reducing and invading them, so they don't think they are as valuable. I think in some of these cases a top human would also be able to invade and live/run if he really had to, but he'd really rather not stake the game on it, whereas LZ etc. are much more confident that they'll find a way.John Fairbairn wrote:LZ didn't like moyo-making moves. Humans did.
My interpretation of the last two points, at east in this game, is that LZ preferred solidity. Humans preferred flexibility. That surprised me. I'd have thought the AI program would prefer flexibility (they "see" farther") and humans tend to go for solidity as a way of reducing uncertainty (and especially so in no-komi games).
Pretty much what I would expect with good play, if Black were aiming to lose by 6.5 pts. or less, given 7.5 komi.John Fairbairn wrote:At move 66, when commentators though the game was no close, the winrate in favour of White had shot up from 54% to over 80%, with no obvious horrendous mistakes - just a slow accumulation of increments. This of course assumes 7.5 komi. I'm guessing that what this means, at least in part, is that as the game progresses so does the [un]certainty and so the program can be ever more confident about its winrate. In other words, it does not necessarily mean Black's play deteriorated. Another 66 moves further on, again with no obvious blunders according to humans, the winrate had increased to 87% (and after a further tranche of 66 moves it was 99.8%).
John Fairbairn wrote:And, for me at least, it remains puzzling what winrate really means, but I infer it matters more in relation to individual moves rather than over a whole game.
It is my impression that the standard value networks are trained with 7.5 komi, so that komi is implied. My guess, however, is that komi must be explicitly supplied for the Monte Carlo playout results. IIUC, the winrates are the average of the two. What happens if the actual komi, zero in this case, is given to the Monte Carlo calculations?dfan wrote:Winrate is how probable LZ thinks it is that Black would win if it were allowed to take over the game and play both sides from here to the end. So certainly if move A has a higher winrate than move B you can infer that LZ thinks that move A is better, without having to worry about exactly what each number means.