It is currently Wed Oct 28, 2020 2:11 am

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 5 posts ] 
Author Message
Offline
 Post subject: KataGo : How many stones ?
Post #1 Posted: Wed Mar 18, 2020 10:17 am 
Lives in gote

Posts: 315
Liked others: 15
Was liked: 94
How many stones could Katago give to a pro ?

There is a short but very interesting thread about KG vs top pros here : HERE
The first (?) LZ network with a pro level was LZ#125 (8000 visits) HERE

I've tried handicap games between LZ#125 and KG. KG wins with many stones, but it may be unfair.
At the beginning of these handicap games, I think LZ sees itself so far ahead, that it plays too conservatively.
Maybe, it would be better to have KG-KG handicap games. But which settings would give KG the same strength as LZ#125 (8000 visits) ?
Kata s297 9d resign threshold 90% 1 thread 16 playouts is 9d , according to @go4thewin.

After some tests :
KG 16p wins ~32% of its games against #125 (8k visits)
KG 50p wins ~75% of its games against #125 (8k visits)
KG 35p wins ~48% of its games against #125 (8k visits)


so, KG 1.3.3 g170e-b20c256x2-s2971705856-d633407024.bin.gz, maxPlayouts=35, numSearchThreads=1 may be around the same level as LZ#125,8kv


KG 1.3.3 g170e-b20c256x2-s2971705856-d633407024.bin.gz, maxVisits=1600 (W)
v.
KG 1.3.3 g170e-b20c256x2-s2971705856-d633407024.bin.gz, maxPlayouts=35, numSearchThreads=1 (B)
H3, komi 0.5
W wins 24-11


KG 1.3.3 g170e-b20c256x2-s2971705856-d633407024.bin.gz, maxVisits=8000 (W)
v.
KG 1.3.3 g170e-b20c256x2-s2971705856-d633407024.bin.gz, maxPlayouts=35, numSearchThreads=1 (B)
H5, komi 0.5 (2 gpus)
W wins 6-4

Unscientific, but fun !

Details (H5 games) :
Attachment:
k1.jpg
k1.jpg [ 464.72 KiB | Viewed 1058 times ]
H5 games :
Attachment:
KG35p_KG8000v.zip [9.08 KiB]
Downloaded 45 times


This post by Vargo was liked by: go4thewin
Top
 Profile  
 
Offline
 Post subject: Re: KataGo : How many stones ?
Post #2 Posted: Wed Mar 18, 2020 6:19 pm 
Lives in gote

Posts: 567
Liked others: 90
Was liked: 623
Rank: maybe 2d
Upcoming in next release is a change that may improve how hard it is to beat KataGo on low playouts when it *receives* handicap stones. The gist of the change is that when receiving handicap stones, Kata will assume that the opponent must be stronger, and use what the neural net has already internally learned from its handicap game training to play more conservatively given that the opponent is believed stronger.

This is exactly analogous to how when giving handicap stones, Kata will play more aggressively to try to keep the game complex in a way that a weaker opponent might find more challenging. The release will simply enable the other side of this behavior for when receiving stones.

Given that Kata is not yet doing the conservative-play side of this behavior, but that human pros are likely to do so to some degree when receiving stones (assuming they are playing to win), probably tests like this overestimate the plausible handicap. And they still might, even after this change - because as I've mentioned before, Kata's modeling of conservativeness/aggression in handicap games is still pretty basic.


This post by lightvector was liked by: Vargo
Top
 Profile  
 
Offline
 Post subject: Re: KataGo : How many stones ?
Post #3 Posted: Thu Mar 19, 2020 3:34 am 
Dies in gote

Posts: 61
Liked others: 12
Was liked: 18
Universal go server handle: Jæja
@lightvector: how well is KataGo able to deal with large reverse komi in the current and new situation? For example, 25.5 reverse komi to replace a 5-stone handicap as described in this table.

Reverse komi is a really nice alternative for handicap games and games turn out way different. If this conservative style were an option, instead of inferred from the fact that a handicap games is being played, then this would be applicable to reverse komi also, I'd reckon. What do you think about this?

Top
 Profile  
 
Offline
 Post subject: Re: KataGo : How many stones ?
Post #4 Posted: Thu Mar 19, 2020 5:05 am 
Lives in gote

Posts: 315
Liked others: 15
Was liked: 94
lightvector wrote:
Upcoming in next release is a change that may improve how hard it is to beat KataGo on low playouts when it *receives* handicap stones.
On Discord, you uploaded a "katago.exe" that should play better when receiving stones, if I understood correctly. And you wrote :
" people should feel free to play around more. kata one-playout or low playouts may become a little harder to defeat when receiving stones now.
I'd also be curious about exactly how the the play is different - i.e. what sorts of moves actually change, that are considered "safer" against a strong opponent. I haven't had time to take a look at that in much detail.
(enjoy!)"


I do enjoy ! Thank you very much !

I reran the exact same handicap tests, with the same commands, only with the "new" katago.exe as black.

The difference is stunning

KG 1.3.3 g170e-b20c256x2-s2971705856-d633407024.bin.gz, maxVisits=1600 (W)
v.
KG 1.3.3 (discord version, 2020.03.19) g170e-b20c256x2-s2971705856-d633407024.bin.gz, maxPlayouts=35, numSearchThreads=1 (B)
H3, komi 0.5

B wins 28-7


KG 1.3.3 g170e-b20c256x2-s2971705856-d633407024.bin.gz, maxVisits=8000 (W)
v.
KG 1.3.3 (discord version) g170e-b20c256x2-s2971705856-d633407024.bin.gz, maxPlayouts=35, numSearchThreads=1 (B)
H5, komi 0.5 (2 gpus)

B wins 10-0


So, I tried a more reasonable H3 game
KG 1.3.3 g170e-b20c256x2-s2971705856-d633407024.bin.gz, maxVisits=8000 (W)
v.
KG 1.3.3 (discord version) g170e-b20c256x2-s2971705856-d633407024.bin.gz, maxPlayouts=35, numSearchThreads=1 (B)
H3, komi 0.5 (2 gpus)

B wins 6-4

PS. In the last test, two games were B+0.5, and in the config file, I have whiteHandicapBonus = 0 , score may have changed if it was whiteHandicapBonus = N (?)

Details for the H3 games
Attachment:
k1.jpg
k1.jpg [ 461.74 KiB | Viewed 906 times ]
The games :
Attachment:
KGhandi35p_KG8000vh3.zip [10.33 KiB]
Downloaded 41 times

Top
 Profile  
 
Offline
 Post subject: Re: KataGo : How many stones ?
Post #5 Posted: Thu Mar 19, 2020 6:35 am 
Honinbo

Posts: 10392
Liked others: 3468
Was liked: 3301
Jæja wrote:
@lightvector: how well is KataGo able to deal with large reverse komi in the current and new situation? For example, 25.5 reverse komi to replace a 5-stone handicap as described in this table.


The table is way off. Even in the old days professionals thought that a handicap stone was worth 10 pts. So for the equivalent of a five stone game they would give 50 pts. reverse komi. We now know that that is too low.

In 1976 I did some statistics on pro-pro handicap games and found a value of about 13¾ pts. per stone. OC, that value cannot hold for reverse komi for high handicaps, but the relationship is approximately linear through 9 stones, which was the limit for pro-pro handicap games. The first stone should be worth half that, or approximately 7 pts. And, as far as we can now tell, that seems to be the case. So proper reverse komi instead of giving 5 stones should be around 60 pts., not 25.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

My two main guides in life:
My mother and my wife. :)

Everything with love. Stay safe.

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 5 posts ] 

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: And and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group