KataGo Distributed Training and new networks

[And]

@akigo
could you evaluate the accuracy of the optimized version of KG 40b? maybe about?
EDIT I mean, how does the level of strength of this network correspond in relation to the original

[go4thewin]

Who cares about accuracy? The most important thing are the tests that you have posted. The nets keep getting stronger. He published the nets, so someone can figure out how to do the same thing. A few months from now, if someone optimizes the strongest confidently rated net at that time, it will be as strong as the nets being released now. That's a really great thing. Before he published the nets no one knew how to do this. There is only a 3-4 month lag in strength for an enormous increase in speed. He has actually made the net stronger at time parity than the unoptimized net s580, but katago is strong enough that the strength differences are irrelevant as long as the nets keep getting stronger. Even if the distributed run stalls, the 40x384 net can take over and then be used to train the 40x256 net. Then the smaller net can be optimized in the end. You will end up with an optimized net stronger than any currently available. Probably with a policy network that will beat the vast majority of amateurs. In the end we have a 48 mb net that is as strong as 175 mb nets and as fast as other 48 mb nets.

[And]

if the optimized network matches the strength of, for example, 99%, then the games results are suitable for it too

[go4thewin]

I see what you're saying, after playing through some games with the optimized net (2 wins 3 losses against s503, white won all the games) I think maybe the best way to get a net that strong would be if a katago run trained a 50x320 net, and then that net was optimized by either akigo or someone on github. But hopefully I'm wrong. maybe better for the end user to just use the optimized net until they can beat it at kyu rank -8, then switch to s580 non optimized. At the lower kyu ranks, the optimized net is really good because it doesn't like to tenuki or play unconnected to either it's own or opponent's stones. So in the kyu rank bots, which tend to tenuki too much with the normal nets, with the optimized net it is lessoned and plays very sensibly all the way down to kyu_rank 10. I think because of the extremely small size of the final net, it would be hard or impossible to make it much more accurate than it already is. I think that's how we get the amazing speed.

[And]

is there a position where the moves of the original s580 and the optimized one differ?

[akigo]

@akigo
could you evaluate the accuracy of the optimized version of KG 40b? maybe about?
EDIT I mean, how does the level of strength of this network correspond in relation to the original

Whenever I release an optimized network (either within the app or outside), I run a series of test matches with 300 playouts between the original net and the optimized one. All these series ended roughly even (+/- 1 game).

When I read your last posts, I thought that perhaps it's different with 1 playout. So I ran another series between original s580 and optimized s580 with 1 playout. After 60 games the score was 31:29 in favour of the original s580.

All these results strongly suggest that the strength difference between original and optimized s580 is pretty small (though you would need many more matches to quantify it).

But what I learnt from this last match is that statistical variance is much bigger with 1 playout: The original s580 started with an 8:2 run in the first ten matches ...

The kyu rank bot further increases the variance, since even at high levels it still considers only a random subset of moves, so there is another element of chance here. So keep in mind that you would need a really huge number of matches to confidently judge its strength.

[go4thewin]

@akigo On your own PC, do you have a way to use the optimized net with a gtp program like gogui-twogtp? If so, would it be possible to upload that net? Even if it is slower that way, it would be so much fun to test it against other bots. Either way, Thanks for the program!

[And]

@akigo On your own PC, do you have a way to use the optimized net with a gtp program like gogui-twogtp? If so, would it be possible to upload that net? Even if it is slower that way, it would be so much fun to test it against other bots. Either way, Thanks for the program!

many @akigo solutions are convenient and simple! there is probably a special version of BadukAI that does this

[akigo]

@akigo On your own PC, do you have a way to use the optimized net with a gtp program like gogui-twogtp? If so, would it be possible to upload that net? Even if it is slower that way, it would be so much fun to test it against other bots. Either way, Thanks for the program!

Currently, using the optimized nets is only possible in the android app. For automated matches you could try running BadukAI in an emulator on PC und use the tools that And described in his very interesting thread "GoAutomation" (they seem to use mouse events and image analysis for communication instead of gtp, so this might also work with the emulator).

[go4thewin]

Ah I see, thanks for running so many games. Just two last questions. 1) For example, I played this game that the optimized net policy beat zen 6 7 dan as white with no komi. But since winrate is not measured when using just 1 playout on my device, I hit ponder alot to see who was winning. Does this give katago more visits, or does it still just use the policy output?
2) It looks like a 60b net is being trained. If hypothetically someone were to make an optimized version of that net using the exact same methods you used, what would the size and speed be? Is it oversimplication to say that since 60*256 is 1.5 times 40*256, the size will be about 48*1.5 mb, so it will be about the size and speed of the non optimized 20b net?
Thanks!

[And]

KataGo s603 playouts 1 - Zen 9d H3 1:1

[go4thewin]

Thanks And for the zen games! I think t4 makes zen weaker than 9d though, because of parallelization. Breakwa11 uses t1 https://github.com/breakwa11/GoAIRating ... /AIcmds.md
But very grateful for the games

[And]

I asked a question about this almost 2 years ago
https://github.com/breakwa11/GoAIRating ... -445428245

[And]

Zenith 7 cpu usage

[And]

it is interesting that on a Ryzen 5, which has 12 threads, Zenith 7 uses only 4