New AI Computer

[RobertJasiek]

Today, I have started serious opening study with KataGo. This lets me be so astonished that I cannot close my mouth even minutes after interrupting study. Opening theory as I knew it almost vanishes. Yesterday, I expected to see the opening revolution but today it is even overwhelmingly revolutionary. AI play is very different: usually territory-orientated unless influence-orientated then necessarily due to global positional context; flexibel unless reinforcing or accidentally agreeing to known josekis. Usually, 5-3 and 5-4 do not exist and 3-3 is depreciated unless globally optimal.

[Knotwilg]

Today, I have started serious opening study with KataGo. This lets me be so astonished that I cannot close my mouth even minutes after interrupting study. Opening theory as I knew it almost vanishes. Yesterday, I expected to see the opening revolution but today it is even overwhelmingly revolutionary. AI play is very different: usually territory-orientated unless influence-orientated then necessarily due to global positional context; flexibel unless reinforcing or accidentally agreeing to known josekis. Usually, 5-3 and 5-4 do not exist and 3-3 is depreciated unless globally optimal.

Well, it will be good to get the view by a leading and thorough theoretic like you. Your brief doesn't surprise me: Uberdude popularized AI opening theory with his "Leela Zero Gospel". Corners then sides then center has become Corners ......................................... then side then center. The value of influence must be clear, often already working with existing influence at the opposite corner. Likewise a pincer is a good option (only) when backed up by influence, making it a "double purpose" move. I haven't read anything by O Meien (?) or Shimure (?) but I believe they've covered those points in detail. But your mind is known - by me at least - to work yet a little differently, so I'm very interested in what you find.

[Cassandra]

Today, I have started serious opening study with KataGo. ...

I would like to recommend you to report the results of your investigations in a new, separate thread.
This is because they have no relation to the hardware or software configuration of your new desktop computer.

[RobertJasiek]

$$B
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . X X a . . . . . . . . . . . . . . |
$$ | . . . O O . . . . , . . . . . , X . . |
$$ | . b . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . O . . . . . , . . . . . , O . . |
$$ | . . . . . . . . . . . . . . X . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------

Click Here To Show Diagram Code

[go]$$B
$$ ---------------------------------------
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . X X a . . . . . . . . . . . . . . |
$$ | . . . O O . . . . , . . . . . , X . . |
$$ | . b . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . , . . . . . , . . . . . , . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . O . . . . . , . . . . . , O . . |
$$ | . . . . . . . . . . . . . . X . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ | . . . . . . . . . . . . . . . . . . . |
$$ ---------------------------------------[/go]

I have used this position as a first test of having 64GB RAM by occupying more than 32GB. I have not measured time exactly but let the GPU work for roughly 1h. The result is as follows, where the hardware is reported by HWiNFO64.

Code: Select all

Move A

39.1%
16.0m visits
6.7 score

Move B

39.0%
8350k visits
6.8 score

RAM

38,784 MB virtual used 
57,235 MB virtual available 
56.3% used 
38,684 MB physical used 
58,278 MB physical available 
59.8% used 

VRAM

11,515 MB available 
   767 MB used 

Note that move A has the higher percentage but move B has the larger score with Black to move. What does this mean for KataGo's own thinking?

I have not watched permanently but noticed a peak of ca. 15,000 visits/s shown in LizzieYZY. Typical values were closer to 4,000. Judging from other positions, speed strongly depends on the position and currently explored variations.

From my usage so far, I can say that I am happy with my decision to buy an RTX 4070 Desktop dGPU. While faster is always nicer in this respect, the card is fast enough to achieve what I want. A purchase of a slower dGPU (such as in a notebook) with roughly 50% speed would also have worked but I would have got the feeling of it being too slow for me. Quite a few people say that every rather modern dGPU, such as 1080TI, would be fast enough but I have to disagree. As a 5 dan and researcher, I really do need such a speed as the 4070 provides.

One must not always trust early values of moves shown in a program. It is sometimes necessary to wait some seconds or maybe minutes. More territorial moves are appreciated earlier than less territorial moves, which need more AI pondering.

[Cassandra]

Code: Select all

[u]Move A[/u]
39.1%
16.0m visits
6.7 score

[u]Move B[/u]
39.0%
8350k visits
6.8 score

Note that move A has the higher percentage but move B has the larger score with Black to move. What does this mean for KataGo's own thinking?

NOTHING!!!

39.0 = 99.74424 % of 39.1
6.8 = 101.49253 % of 6.7

[Cassandra]

One must not always trust early values of moves shown in a program.

It is often helpful to go back one move and forward again during the run of the analysis.

[RobertJasiek]

Yes, going back and forward I have already been using to compare values.

NOTHING!!!

39.0 = 99.74424 % of 39.1
6.8 = 101.49253 % of 6.7

It means 0.25576% or 1.49253% loss during just one move. This is much considering that a game has many moves. These percentages do not translate linearly to such we humans perceive but still I do not ignore every small number. For lesrning, I'd say both moves are about equal. If, however, other two moves have slightly larger value differences, my alarm bells ring.

BTW, if the upper right corner stone is Q16, both moves have the same percentage and empirical score in a quick check.

[kvasir]

Note that move A has the higher percentage but move B has the larger score with Black to move. What does this mean for KataGo's own thinking?

The search will spend more most of the time searching the 'best' move, how many playouts each move gets is a good indicator if it is considered the best move even if the difference is tiny.

When I analyze the same position with komi the difference between E17 and B15 is no much.

E17
623k
0.0 pts. change
+0.0%


C16
79.0k
-0.15 pts. change
-1.8%

B15
298k
0.0 pts. change
-0.1%

You could say that E17 is slightly favored over B15 by KataGo, the evaluation is very close but it keeps searching E17 more. Either move is clearly favored over C16. One goal of opening theory is to avoid playing moves like C16 when better moves can easily be identified; it is not that it is so much worse it is that we know it is worse

A common human thinking patterns seems to be to assume that words like "always" are implied when they are not, therefore the conclusion could be "[everyone] [always] assume KataGo [always] dislikes C16 in [every] similar positions [and all the time]". Jumping to conclusions like that is not warranted, more investigation undoubtedly could find positions were C16 is preferred. I think that especially applies for moves in quiet positions that are evaluated just slightly worse, but if we 'know' it is worse in a particular position that might then be all we really care about.

[Cassandra]

If it is your desire to find the "best" move in a given position, you will probably have to adjust the komi until the win-rate is in the 50% range.
KataGo is best if it thinks that the game is a close one (which your example position is no longer).

As far as I know, you can adjust the komi in the middle of the running analysis via the command line ("komi=x").

[RobertJasiek]

Ah, I have been in need of the komi setting hint, thanks!

C16: I wondered the same and constructed some position in which it is the best move according to KataGo:)

[Harleqin]

It means 0.25576% or 1.49253% loss during just one move. This is much considering that a game has many moves.

It should be noted that even completely symmetric options often yield such differences. The noise is bigger than you might assume.

[RobertJasiek]

you will probably have to adjust the komi until the win-rate is in the 50% range. [...] you can adjust the komi in the middle of the running analysis via the command line ("komi=x").

Still I try to figure out how to adjust komi.

You mention a command line. Do you mean every command line in every GUI for a particular instance of KataGo? If so, does one add, say, komi=7.5 as another parameter or must this preceded by another gtp or analysis command, somehow as follows?

Code: Select all

<path>\<katago_file_name>.exe gtp -model <path>\<model_name>.bin.gz -config <path>\<gtp_file_name>.cfg komi=7.5

LizzieYZY seems to have three kinds of settings affecting komi:
- KataGo setting: show a) score or b) score & komi. Initially, (a) leads to -0.1 while (b) leads to ca. 6.6 for the score values on the stones.
- Engine setting: for each command line without any komi=7.5 parameter and area scoring, the list of engine settings has a Komi column with 7.5 in each row. This and some KataGo description of using such a komi by default lets me believe that this komi is being used.
- Komi field in the GUI menu bar: its value 7.5 results in roughly 38% percentages; its value 7.0 results in roughly 50% percentages. Now, I prefer the latter. However, what does it mean? Does KataGo still use the 7.5 of its engine command interpretation or does it then use 7.0? Or does the komi field in the GUI menu bar only affect the displayed values on the stones?

Currently, I use LizzieYZY in permanent gtp pondering mode without starting play against the engine. Permanently, I watch the top move values (percentages, visits, scores) on the stones and click whichever next move I want. I do not select any analysis command and LizzieYZY does not use analysis command lines.

So how do I ensure that, say, 7.0 or 7.5 komi is used while initially seeing close to 50% and 0.0 scores? Am I already doing it right now by
- not adding any komi parameter in the engine command line,
- seeing the engine settings stating 7.5 komi being used,
- selecting the different 7.0 komi in the komi field in the GUI menu bar and
- using the GUI's KataGo setting to show only the score?

[Cassandra]

I am sorry, but I do not have any experience with LizzieXYZ.

Tried the komi-command-line-issue in Sabaki, but it did not work as expected.

However, it is easy and quick to determine the "close" komi in Sabaki:

-- Attach your engine used for the first time via "Engines" > "Attach" > "Select-in-the-drop-down-lists". Set the komi e.g. to 7.5 (if this is not the default value).
-- Start analysis via "F4". (let's assume a win-rate of 39.x displayed).
-- Attach your engine used for the second time, but use a smaller komi, e.g. 6.5.
-- Start analysis via "F4". (let's assume a win-rate of 49.x displayed) ==> finished; use this komi in LizzieXYZ (if possible therein).

Otherwise:
-- Attach -- Analyse -- Repeat

[RobertJasiek]

After playing around with komi in LizzieYZY, it seems to work as follows:

- Every engine setting has its komi, which is loaded together with the engine.
- Later, the GUI komi value can be altered to override the initially loaded value.
- For area scoring, 7.0 komi gives closest values initially near (but not exactly at) 50% and 0.0 score.
- To see these values, the LizzieYZY KataGo setting for values displayed on stones should only show score (instead of score + komi). This nomenclature is strange, but so it is.

In general, I guess that every GUI might need some playing around with komi values until the behaviour is as desired (see above).

[lightvector]

If you're studying theoretic openings, then sure, go for the fairest initial komi. That should be 7.0 with area scoring, and 6.5 with territory scoring.

If you're analyzing a real game though, and the real game actually used komi of 7.5, rather than 7.0, for example, I wouldn't bother adjusting the komi merely to switch between 38% and 50%. There isn't going to be a meaningful difference in play quality there. Mainly it's going to have an effect for winrates that are much more extreme. KataGo's analysis quality also doesn't diminish much in imbalanced games compared to older-generation AlphaZero bots that operated on winrate alone and didn't care about score. Even when far ahead or far behind, it will still suggest good moves.

Note that adjusting the komi will affect the answer you get back. For example, if you adjust the komi to make the winrate 50% instead of 97%, KataGo might in some sense be trying for more "optimal" lines due to treating the game as being close and needing to extract every point. But it might also avoid suggesting a simple line that in the *actual* game, with the *actual* komi, would have been a safe win that gives up 1 point but still leaves you ahead by 10 points, in favor of starting a huge messy complicated tactic that even KataGo isn't sure about the result of. It might occasionally even be the case that the simple tactic seeming to give up a point is more correct, if the complicated tactic is misevaluated.

So it depends on what you want. Even professional players often try to play a bit more solid and give up tiny fractions of a point when ahead, or to start complications when behind, and KataGo will do the same to a degree, so if you're trying to get practical analysis of a real game, you might prefer analysis to have those very mild biases baked in, in which case you wouldn't adjust komi, or would only adjust it partially. You might adjust komi fully if you wanted specifically to study the variability of move choice with komi, or if you were trying to find the theoretical sharpest and least-forgiving ways to play.

And yes, for LizzieYZY there's a setting where it reports the on-the-board score without komi being accounted for, and the naming about which one is which is weird. (But as you found, it's easy to tell by trial and error, the setting that reports the starting board being 50% winrate and yet 7 score is of course the on-the-board score mode).