Tenchou no Igo 7 Zen

[Cyan]

Zen 7 is really strong, even though it only uses cpu. Here are two games against Leela 0.11.0 and AQ 2.0.3. Hardware: i5-2500K, GTX 1060

[Bas]

I let play Zenith Go 7 against Leela 0.11.0 GPU with 120sec per move. 4 games, 3 won by Zen.

The assumption was that the longer thinking makes the advantage of the GPU more obvious, but even than Zen played very well (except for one game where it played odd moves). Leela has blind spots and a weakness in evaluating positions. Zenith 7 plays more consistent and has a very nice style which is more related to Japanese go theory. I think its NN is trained and supervised carefully and has a much higher quality than Leela's NN.

My hardware specification again:
i7-7700@4.2GHz
MSI GeForce GTX 1080 Gaming X 8G (but not overclocked)

Leela has used more thinking time, Zen often didn't used the 120 seconds. But I only have the average thinking time of Zen.


Zen's average thinking time: 63.4 sec/move


Zen's average thinking time: 64 sec/move


Zen's average thinking time: 62.3 sec/move


Zen's average thinking time: 67 sec/move

[Caesura]

Just some unorganized notes for anyone who might be interested in buying Zen 7 and looking for info:

By my understanding and impressions of both, Zen 7 is significantly stronger than Leela 0.11 when using equal hardware, but Zen 7 commercial release doesn't support GPU, unlike Leela. So if you have a laptop like mine without any good graphics card anyways, Zen 7 is definitely stronger, but if you have a decent graphics card, Leela might well be as strong or stronger because Zen will be unable to use that hardware.

On CPU, stylewise Zen 7 so far seems on average a little more stable and self-consistent in its evaluations and very steady. Leela is more dynamic and prone to change its mind as you play down a variation, and continues to have occasional blind spots or misevaluations just like the older 0.10 Leela, although they are somewhat less frequent. I don't have any experience with GPU-accelerated Leela, but I would expect with the large speedup of a strong GPU it would continue to have rare large misevals in situations where the search isn't sufficient to correct it, but I could easily see the deeper search making it much sharper and competitive with Zen 7 in fights that it doesn't miseval.

Zen 7 doesn't appear to have a nice "offline" review interface right now, or at least I haven't found it, but I've found it okay for interactive review of games. The interface is a bit lacking through. By contrast, with Leela 0.11 GTP version there are scripts you can find online that will let you do things like have Leela deep-search your game SGF overnight and annotate it with what it thinks your mistakes were, as well as plug them into GUIs that let you navigate and make reviewing easy.

However, with Leela I always have to take its evals with a grain of salt when they get in certain kinds of capturing races or life and death due to its blind spots. From experience I have a sense now of when it's more likely to do this and in those cases will often interactively go to the position and "test" Leela by playing the moves to resolve a situation or that I think might expose a blind spot, before I can trust the eval. I've used Zen less, but I haven't noticed such major problems with Zen.

Also, one last detail - Leela has an interesting behavior when reviewing handicap games. For example, in no-komi games it thinks *white* has the advantage early on, and in low-handicap games it thinks Black has none or only a small advantage. This is presumably because it was trained on human games where in practice, no-komi games often are won by white since they are underhandicapped, etc. But this makes it hard to use Leela to "objectively" evaluate the position sometimes.

And yet I don't think you can always rely on it to give you a reasonable value of things like "how likely is black to win given that he is 2 stones weaker than white" either. Because from what I can tell from what gcp has said, Leela doesn't actually know that for example black is 2 stones weaker than white. It simply infers this from seeing the board position with black having the extra hoshi stone. But as the board fills up and captures happen, it's no longer clear from the board position alone that it was originally a handicap game, so I think Leela in some unknown and hard-to-predict way will gradually transition to no longer evaluating as if the players are different in strength.

Zen seems to report values closer to objective, at least in 2 and 3 stone games unlike Leela it reports a strong advantage for Black at the start. I haven't tried using Zen on a no-komi game yet.

Thanks for this very useful post. Wish I had read it before I bought zen7.

I think what you call "a nice "offline" review interface" was what I had in mind when Zen7 was advertised as having a "review mode". Its a bit disappointing that there is no such option available, when freely available software (eg Leela + GRP) can do so. It is really time consuming to review a game using the analyze function alone. At the very least Zen 7 should be able to highlight the moves that most require scrutiny: there is a similar function when reviewing games played against zen7, where it highlights moves that require review in red.

As things stand, i am not sure whether to review my games with Zen or Leela+GRP (i dont have GPU). Perhaps the choice will in time be an obvious one, once Leelazero powers up

[Gomoto]

I prefer Zen to Leela for this feature:

With Zen I can add and explore multiple variations and annotate them on the fly.