Developing weak AIs in KaTrain

[jann]

Changing komi seems similar to simply printing the bot's evaluation (incl. expected score) each turn - same feedback.

I recall my brother several years ago, who played a few games with a chess engine for fun. He said afterwards that he won't do this again, because of those evaluation numbers that were printed after each move - and which kept monotonely changing to the program's favor regardless of which move he played.

[jlt]

Try 'P:weighted' and see how it feels.

I just played a couple of quick games. In the first one I resigned quickly after losing a group. I won the second game by 18.5. The game felt pretty natural for about 150 moves, but the AI started playing strange moves when it was behind, wasted ko threats although a ko was going on, and didn't see immediate threats at the late endgame.

[Bill Spight]

Changing komi seems similar to simply printing the bot's evaluation (incl. expected score) each turn - same feedback.

Well, yes, it is. However. . . .

I recall my brother several years ago, who played a few games with a chess engine for fun. He said afterwards that he won't do this again, because of those evaluation numbers that were printed after each move - and which kept monotonely changing to the program's favor regardless of which move he played.

Unlike with chess, changing the komi doesn't simply give the human feedback, it also changes the chances of winning the game, in the human's favor. Unless the human is very, very, very strong.

[Bill Spight]

OK, here is an idea for a weak AI of SDK strength.

Train a policy network on 10 kyu games. But use the bot's regular MCTS and value network for search and evaluation of leaf nodes. The bot will choose candidate plays like weak humans, but evaluate them well. Depending upon the amount of search, that should produce a bot that plays at SDK levels.

[sanderl]

OK, here is an idea for a weak AI of SDK strength.

Train a policy network on 10 kyu games. But use the bot's regular MCTS and value network for search and evaluation of leaf nodes. The bot will choose candidate plays like weak humans, but evaluate them well. Depending upon the amount of search, that should produce a bot that plays at SDK levels.

That would certainly work, I think lightvector trained a net once that could predict moves at all levels. Training such a net would take more time though!
A similar approach is using a 6b net from the middle of the training run, I'm waiting for the training run from katago to be fully published before trying that, but some from the old run look promising. They're not quite human errors though.

[sanderl]

Try 'P:weighted' and see how it feels.

I just played a couple of quick games. In the first one I resigned quickly after losing a group. I won the second game by 18.5. The game felt pretty natural for about 150 moves, but the AI started playing strange moves when it was behind, wasted ko threats although a ko was going on, and didn't see immediate threats at the late endgame.

Yeah wasting ko threats is a very persistent issue. I guess atari/throw-in is fairly high up there in terms of policy and the score loss of ko threats is fairly minimal.

[sanderl]

There's some interesting work going on in this github issue.
A contributor developed some calibration for playing a reasonably even game against gnugo and pachi, leading to a new AI option. However, the next step is even more challenging: can we invert this and estimate someone's rank from their moves automatically?
I'd be interesting in hearing if any of you have ideas about this.

[Bonobo]

Just want to note that I just installed KaTrain on my iMac (June 2017, 27" Retina 5K, macOS 10.14, 32 GB RAM), installation was a breeze following your installation instructions, just a few Terminal commands copy/pasted, and your Teaching Game option simply blew me off my feet, this is SO! AWESOME!
Very nice also that I can use it to play against a calibrated version of KataGo, so I can use this very strong AI to play against an opponent who is just a little bit stronger than I am while learning better moves. (Of course I still prefer human opponents for “real” games but as a tool KaTrain is wonderful!)

I immediately decided to sponsor your project (albeit with a small amount only) on https://github.com/sponsors/sanderland, as I think this is an extremely valuable tool for learning.

Before, I hesitated to install KataGo etc. because I was afraid it would be too much fumbling, but KaTrain makes it all very easy.

Screenshot:

83739443_10220673464856795_7731822805852681469_o.jpg (81.58 KiB) Viewed 16089 times

[Javaness2]

Just out of interest; if you take a weight file from a STRONG neural net and change just some small part of its contents, what will that do to its strength? Is it a sick hack too far?

[Bonobo]

Sander,

I’m not sure where the best place is for a support question, so I’m asking it here:

Re: “models”

Screen Shot 2020-06-29 at 18.17.34.jpg (30.87 KiB) Viewed 16038 times

I’m a total n00b when it comes to using modern Go AI apps, understanding close to nothing of “weights”, “playouts”, visits, “depth” and “width”, etc.—how do I know which the “best” model for which purpose?

TIA,
Tom

[lightvector]

Here's some guidance on that question (scroll down a little from here to the "Which model/network should I use?"). Anyone should feel free to relay this guidance. You might have to drill down into the links there to see which models match up with which models that katrain is showing you, but the general point is that bigger nets are slower to run but stronger. Often stronger enough to compensate or more than compensate being slower, but depends on hardware.

https://github.com/lightvector/KataGo#other-questions

[Bonobo]

Thanks a lot, @lightvector!

[Pio2001]

Hi,
Just my two cents : after a bot is weakened, it would be a good thing to prevent it to ignore important ataris.

It feels really unnatural and disappointing when you play atari to a huge chain of stones and the bot just doesn't see it. Capture, game over.

[sanderl]

Hi,
Just my two cents : after a bot is weakened, it would be a good thing to prevent it to ignore important ataris.

It feels really unnatural and disappointing when you play atari to a huge chain of stones and the bot just doesn't see it. Capture, game over.

It tries to do this by making 'important enough' moves non-optional, but it can be hard to determine which ones are important enough. However, a super hard to see 30 point life and death problem and a 15 stone atari can be essentially equivalent.
Also when this is a bit too highly tuned, it's easy to get an AI which is accidentally high dan.
If you save example games in which this happens and send them, it will help in creating something that might avoid this.

[Uberdude]

How about making weak KataGo not playing a move worse than what low level GnuGo would pick. That would probably be a reasonable way to discard wacky bad moves a 25 kyu wouldn't do.