Easy to set up Go AI?

[tj86430]

If you are on Windows

What about those who are on MacOS?

Lizzie / Leela Zero works on Mac too, but you need to compile it yourself (which Tami requested not needing to do), there are instructions in the mac release on the page I linked. Quite why there can't be a pre-built executable I don't know, is each Mac really so different? "Mac: It just works"

Thanks. In the complilation of Leela Zero there is a bunch of "deprecated" warnings, but no errors.

Lizzie starts and displays "Leela Zero is loading..." for a while and then exists with message "Leelaz process ended."

Probably one should start by trying to eliminate those warnings (they all seem to be OpenCL-related, so perhaps just disabling OpenCL in config.h might help)

[Tryss]

You should verify if LZ works without lizzie, if it does, the problem is with lizzie, if it don't, the problem is with LZ

[tj86430]

You should verify if LZ works without lizzie, if it does, the problem is with lizzie, if it don't, the problem is with LZ

I will do that iff I decide to invest more time into this.

[Tryss]

Just your terminal, go to your lz folder and type

leelaz --weights=yourWeightsFile --benchmark

[tj86430]

Just your terminal, go to your lz folder and type

leelaz --weights=yourWeightsFile --benchmark

Ok, I need a weights file.

After a bit of googling I downloaded one from http://zero.sjeng.org/networks

After that I get:

Failed to enqueue: 290 kernels
Failed to find a working configuration.
Check your OpenCL drivers.
Minimum error: 100.000000. Error bound: 0.100000
Failed to enqueue: 290 kernels
Failed to find a working configuration.
Check your OpenCL drivers.
Minimum error: 100.000000. Error bound: 0.000100
libc++abi.dylib: terminating with uncaught exception of type std::runtime_error: Tuner failed to find working configuration.
Abort trap: 6

Probably need to compile again without OpenCL

[tj86430]

With a weights file AND recompiled without OpenCL it works.

[Knotwilg]

Since this is a free and responsive help desk, I thought I'd report on my issue here

I downloaded both zip folders CPU and GPU, not knowing the difference. Then I unzipped them and launched the application. In both cases a command prompt like window popped up and promptly disappeared.

OK, now what?

Thanks guys

[Uberdude]

launched the application

means double clicked leelaz.exe? You should double click Lizzie.jar and, assuming you have .jar files associated correctly, that should open the lizzie interface. If not open a command prompt in that folder and run "java -jar Lizzie.jar".

[Knotwilg]

launched the application

means double clicked leelaz.exe? You should double click Lizzie.jar and, assuming you have .jar files associated correctly, that should open the lizzie interface. If not open a command prompt in that folder and run "java -jar Lizzie.jar".

My saviour!

Yes, I was born and raised on IBM DOS, managed to get through to Windows and when the java era came around, my intuition had already stopped developing.

Edit: I analyzed my first game using Lizzie and I can feel signs of an upcoming addiction.

[John Fairbairn]

First a comment. I have mentioned several times before that pros usually don't count the way we do. They can look at the board and see inefficiencies and simply count those (there aren't so many in pro games). Generally the side with the most inefficiencies is behind, but obviously some inefficiencies are worse than others and pros seem to have enough skill to assess how much each one is worth (I expect, though, some are significantly more skilled at this than others). Because they have this feel for the game, they stress efficiency of plays an awful lot.

I've understood this, and have even been able on occasion to simulate their behaviour, but mostly I have treated it as a little bit of a party trick by them. However, my first forays with Lizzie/LZ have astonished me because it spots inefficiencies very early in the game and marks them down heavily. For example, there was a joseki where a connection was needed and two were available. The pro played the solid connection but LZ preferred a hanging connection. It hardly seemed to matter because there was no immediate danger, no shortage of liberties, nothing special at all - purely a case of long-term efficiency. But LZ adjudged the pro's solid move a whopping 6 percentage points worse. That pattern seems to emerge elsewhere and so I now understand even better why pros care about efficiency of plays, but at the same time it seems they have not entirely mastered all the elements of good shape.

Second, another question. I have read that the AI programs don't work happily with different komis. But presumably this has been discussed. I'd love to know what the general feeling is as to how much less reliable LZ is if the game komi is zero, or more specifically if LZ starts with a winrate of say 46% for Black on an empty board in a no-komi game (i.e. it still believes komi of 7.5 applies), what is the general LZ-assisted human view as to the real winrate knowing that komi is actually 0 (and likewise for various handicaps)?

[dfan]

Second, another question. I have read that the AI programs don't work happily with different komis. But presumably this has been discussed. I'd love to know what the general feeling is as to how much less reliable LZ is if the game komi is zero, or more specifically if LZ starts with a winrate of say 46% for Black on an empty board in a no-komi game (i.e. it still believes komi of 7.5 applies), what is the general LZ-assisted human view as to the real winrate knowing that komi is actually 0 (and likewise for various handicaps)?

Unfortunately, it depends. I would still happily follow LZ's judgment in the early game even though it was mistaken about the size of the komi; maybe it thinks that the winrate is 46% while it is really 60% or something, but it'll still make reasonable moves. On the other hand, at the very end of the game, if Black is ahead by 3 points on the board, say, it will think that Black's winrate is almost 0% when it really should be almost 100%. So I think its endgame evaluations in a close game (as will happen in most pro games) will be pretty useless.

[Bill Spight]

First a comment. I have mentioned several times before that pros usually don't count the way we do. They can look at the board and see inefficiencies and simply count those (there aren't so many in pro games). Generally the side with the most inefficiencies is behind, but obviously some inefficiencies are worse than others and pros seem to have enough skill to assess how much each one is worth (I expect, though, some are significantly more skilled at this than others). Because they have this feel for the game, they stress efficiency of plays an awful lot.

There are other reasons to stress efficiency, I think.

Books about positional judgement start with counting more or less sure territory, but non-territorial players, like Takemiya, have to rely upon other factors, such as efficiency. There is no getting away from the need to assess influence.

I've understood this, and have even been able on occasion to simulate their behaviour, but mostly I have treated it as a little bit of a party trick by them.

There are tricks, such as the saying that ponnuki is worth 30 points. Even the idea that one handicap stone is worth 10 points is sort of a trick. But we now know, because of komi, that a handicap stone is worth around 14 points (to a good player). That's a 40% difference.

However, my first forays with Lizzie/LZ have astonished me because it spots inefficiencies very early in the game and marks them down heavily. For example, there was a joseki where a connection was needed and two were available. The pro played the solid connection but LZ preferred a hanging connection. It hardly seemed to matter because there was no immediate danger, no shortage of liberties, nothing special at all - purely a case of long-term efficiency. But LZ adjudged the pro's solid move a whopping 6 percentage points worse. That pattern seems to emerge elsewhere and so I now understand even better why pros care about efficiency of plays, but at the same time it seems they have not entirely mastered all the elements of good shape.

It takes experience to develop judgement. Modern bots enjoy the experience of millions of games. No human can match that.

Second, another question. I have read that the AI programs don't work happily with different komis. But presumably this has been discussed. I'd love to know what the general feeling is as to how much less reliable LZ is if the game komi is zero, or more specifically if LZ starts with a winrate of say 46% for Black on an empty board in a no-komi game (i.e. it still believes komi of 7.5 applies), what is the general LZ-assisted human view as to the real winrate knowing that komi is actually 0 (and likewise for various handicaps)?

In the 1970s I saw some Nihon Kiin statistics on 2800 pro games, 1400 played with 4.5 komi, 1400 played with 5.5 komi. It was partly on that basis that I predicted that Japanese komi would be 6.5 by the year 2,000. (Almost. ) For both sets of statistics the median result was between 6 and 7 points on the board for Black. Also, IIRC, if you applied different komis around 6.5 to the data, a one point difference in komi meant a difference in win rates of 2% plus. Leela Zero uses area scoring, where minimum score differences are nearly always at least 2 points. So a minimum score difference translates to a winrate difference of around 4-5%. So the pro's mistake (if Leela Zero is right) cost maybe 2-3 pts. on the board.

There are amateur dans players who scoff at 2 pt. mistakes in the opening. But Leela doesn't.

As for the reliability of Leela Zero with different komi, or no komi, I think that win rate differences aren't much affected early in the game. For winrate estimates themselves, maybe add 10-15% to Black's winrate estimate, early in the game. (???)

[Tryss]

There's an experimental 0 komi version of LZ available somewhere, but I never tried it. I think it can be used to analyse no komi games correctly.

[Uberdude]

Second, another question. I have read that the AI programs don't work happily with different komis. But presumably this has been discussed. I'd love to know what the general feeling is as to how much less reliable LZ is if the game komi is zero, or more specifically if LZ starts with a winrate of say 46% for Black on an empty board in a no-komi game (i.e. it still believes komi of 7.5 applies), what is the general LZ-assisted human view as to the real winrate knowing that komi is actually 0 (and likewise for various handicaps)?

2. There is an experimental version of LeelaZero that works for 0 and other komis (but it's not as easy to install). It basically does a trick of switching black and white and going for the middle and hoping the magic of the neural networks is basically linear, and it seems to work fairly well (though not being self-play trained in these conditions means it'll be less accurate). With network #153 (LZ's opinion does slowly evolve with new versions) it thinks black starts off at 61.5% on the empty board with no komi (cf 46.5 with 7.5 komi). I did a little analysis of Shusaku's famous ear-reddening move with it here.

Here's some win% for various handicaps (just pass in lizzie with "p") with network #157 (the best 15 block and the one I mostly use now) at 30k playouts (doesn't change much by then). Note that 157 is higher than 153, as bots get stronger this probably happens, Elf is even higher (it thinks black 2 stones with 7.5 komi is 99%). A perfect bot would obviously say 100% for black first no komi, and we presume 0% for 7.5 komi if it is indeed too high.

Code: Select all

Black starts, white 7.5 komi ("even game"):     46.7
Black starts, no komi                           65.5
Black starts with 2 stones, white 7.5 komi      80.7
Black 2 stones, no komi                         88.8
Black 3 stones, 7.5 komi                        92.6
Black 3 stones, no komi                         93.2
Black 4 stones, 7.5 komi                        96.5
Black 4 stones, no komi                         88.4  (yes, really, don't trust interpolation/extrapolation too much when you adventure to new lands)

[Uberdude]

First a comment. I have mentioned several times before that pros usually don't count the way we do. They can look at the board and see inefficiencies and simply count those (there aren't so many in pro games). Generally the side with the most inefficiencies is behind, but obviously some inefficiencies are worse than others and pros seem to have enough skill to assess how much each one is worth (I expect, though, some are significantly more skilled at this than others). Because they have this feel for the game, they stress efficiency of plays an awful lot.

Well, I rarely count points in games, and usually use the "sum of how bad I think my mistakes were" vs "sum of how bad I think opponent's mistakes were" as the basis for if I think I'm leading or not. I did actually count a few times in my recent title match game: in the middlegame my count suggested playing to defend my 3-3 could be ok but there was just too much play left in other areas of the board I wasn't confident to play a conservative gote move, and then things got out of hand and I never got the chance, and in the endgame I counted it was super close so did some reading instead of being lazy and playing (just!) unnecessary defence. I have noticed that in his commentaries Myungwan Kim counts territory a lot (though often does the "white needs to make at least 15 in this hard to count area to be even" approach) whereas Michael Redmond rarely does.

I've understood this, and have even been able on occasion to simulate their behaviour, but mostly I have treated it as a little bit of a party trick by them. However, my first forays with Lizzie/LZ have astonished me because it spots inefficiencies very early in the game and marks them down heavily. For example, there was a joseki where a connection was needed and two were available. The pro played the solid connection but LZ preferred a hanging connection. It hardly seemed to matter because there was no immediate danger, no shortage of liberties, nothing special at all - purely a case of long-term efficiency. But LZ adjudged the pro's solid move a whopping 6 percentage points worse. That pattern seems to emerge elsewhere and so I now understand even better why pros care about efficiency of plays, but at the same time it seems they have not entirely mastered all the elements of good shape.

I think far too much judgement by pros of josekis (or at least what I've gleaned of it) is based on reference/tewari/comparison (where such comparisons can accumulate asymmetric errors) to other results they think (possibly mistakenly) are ok, instead of trying to make more absolute judgements based on where the stones end up. "Well we played these moves which are probably okay and it started even so it ended even" isn't great. Robert Jasiek's territory/influence stone counting approach is a nice idea in that direction, but just not good enough to be useful. Basically you need a massive function with gazillions of parameters finely tuned to judge a multitude of facets of a position. Hang on a minute, I just described a neural network!

6% is quite a big mistake for a pro, but not that unusual. If you want to see lots of big % drops to shatter the illusion pros know what they are doing then I suggest reviewing pro games with Elf . I was recently reviewing the 5th Kisei title match of Shuko vs Otake with Go world commentary and half the opening moves were 8-12% point mistakes according to Elf (lots of splitting or extending on side moves, even before bots that had gone out of fashion as not active/efficient enough). LZ #157 was much more tame, just rating them -1 to -3%. Here's a graph from LZ (the horizontal dashed lines are at 25/50/75%).

Kisei5g1LZ157.PNG (106.91 KiB) Viewed 9066 times

You can see the little zigzags as they play side moves in the opening. That big drop on move 23 of 10% received no comment. Shuko did criticise Otake's move a few later (LZ agrees it was bad, -3%) and says "This move surprised me. I think it must have been a slip. Black would have had quite a good game if he had connected at 34 instead". LZ thinks the damage was already done and white would still be 70%, though that is with 7.5 not 5.5 komi. By move 35 "... makes it clear how badly Black has done here. The general feeling was that he [Otake] had suffered a loss equivalent to the komi. Some professionals commented that the game had already been decided". LZ with it's 7.5 komi says white was at 76%. Something to note is LZ didn't anticipate Otake's good move 29 beforehand so its judgements preceding that are less trustworthy, but once it gets to that move it does find it after about 30,000 playouts (so non-GPU people will probably never find it). You can compare this graph to the one with my title match review to see the difference in sizes of mistakes between top pros of the 80s and 2 EGF 4ds. (This is based on the assumption LZ is stronger than the players, which is probably true but might not be in some complicated fights or ladders where it can have delusions, though these tend to disappear with sufficient playouts).

Here's the graph from Elf v1. You can see the much bigger swings from opinionated Elf as they play side instead of corner moves. What was -10% to 70% for white for LZ 157 is -21% to white 95% for Elf. White is at 99% by move 35. Elf thinks Shuko did make some mistakes in the middle game which took him to only 80% but that didn't last long.

Kisei5g1Elfv1.PNG (102.25 KiB) Viewed 9066 times

And another LZ157 graph for comparison, a half-pointer from Park Junghwan vs Mi Yuting from a few days ago (at 7.5 komi). The deviations are generally smaller than Shuko vs Otake, so LZ is supporting the belief that top pros of today are stronger than of the past (though in opening they can be playing bot josekis which LZ will approve of (unless they do it in bad situations, which does actually happen a lot) but maybe that's not a caveat if such josekis are actually better ) Btw, white's 40 was the biggest mistake of the opening at -9%, can you find the better move? It's totally logical and a beautiful tesuji that exploits black's timing mistake. LZ is good at this sort of sharpness in fighting. That big spike at move 110 is for the f11/g11 exchange that I couldn't understand and neither can LZ, saying it was bad. I'd really like to know what the players were thinking: did they see something LZ didn't which when shown LZ agrees was important (well done them), or did LZ not see it but when shown it thinks it's bad and sticks to its opinion, or did LZ see and dismiss it as bad? Also some of the big zigzags later I think are due to foibles in LZ's analysis, for example it wants to play the profitable sente exchanges from t13 for white quite early, so when white does something else it sometimes doesn't see (at 10k playouts) that white will be able to play them later too so thinks the other move is worse when really it's just a different order of chunks. Being a close game big swings could just be 1 point mistakes that change the winner. Also it wastes playouts throughout the game wanting to play black's a13 sente move, which humans can save for a ko threat with little thought and then stop wasting brainpower on. And generally I think the fighting might be too complicated for LZ on moderate playouts that I wouldn't confidently say they must be making big mistakes here. With more time I can try to tease out the truth, but really a stronger player than myself is needed for a reliable interpretation (Elf can help).

ParkvsMiLZ157.PNG (415.12 KiB) Viewed 9050 times

Elf v1 on only 1k playouts on whole Park vs Mi. Noticeable that in the early game it prefers black and generally sees a lot more going on. For starters it thinks after parallel 4-4s white answering knight approach with a knight move is a 7% mistake (Elf v0 thought it was fine). Has Elfv1 discovered how to make better use of black's sente advantage? And it thinks Mi only won very late with Park connecting the ko, though I don't know if it can correctly read/intuit the deciding ko fight with few playouts. Update: playing out the ko fight as Elf wants sees the win gradually become white's, so it seems it was mistaken to think black was winning before.

ParkvsMiElf1.PNG (538.4 KiB) Viewed 9047 times