Yeah wasting ko threats is a very persistent issue. I guess atari/throw-in is fairly high up there in terms of policy and the score loss of ko threats is fairly minimal.

There's some interesting work going on in this github issue.
A contributor developed some calibration for playing a reasonably even game against gnugo and pachi, leading to a new AI option. However, the next step is even more challenging: can we invert this and estimate someone's rank from their moves automatically?
I'd be interesting in hearing if any of you have ideas about this.

Just want to note that I just installed KaTrain on my iMac (June 2017, 27" Retina 5K, macOS 10.14, 32 GB RAM), installation was a breeze following your installation instructions, just a few Terminal commands copy/pasted, and your Teaching Game option simply blew me off my feet, this is SO! AWESOME!
Very nice also that I can use it to play against a calibrated version of KataGo, so I can use this very strong AI to play against an opponent who is just a little bit stronger than I am while learning better moves. (Of course I still prefer human opponents for “real” games but as a tool KaTrain is wonderful!)

I immediately decided to sponsor your project (albeit with a small amount only) on https://github.com/sponsors/sanderland, as I think this is an extremely valuable tool for learning.

Before, I hesitated to install KataGo etc. because I was afraid it would be too much fumbling, but KaTrain makes it all very easy.

Screenshot:

_________________
“The only difference between me and a madman is that I’m not mad.” — Salvador Dali ★ Play a slooooow correspondence game with me on OGS?

Just out of interest; if you take a weight file from a STRONG neural net and change just some small part of its contents, what will that do to its strength? Is it a sick hack too far?

_________________
North Lecale

Sander,

I’m not sure where the best place is for a support question, so I’m asking it here:

Re: “models” I’m a total n00b when it comes to using modern Go AI apps, understanding close to nothing of “weights”, “playouts”, visits, “depth” and “width”, etc.—how do I know which the “best” model for which purpose?

TIA,
Tom

_________________
“The only difference between me and a madman is that I’m not mad.” — Salvador Dali ★ Play a slooooow correspondence game with me on OGS?

Here's some guidance on that question (scroll down a little from here to the "Which model/network should I use?"). Anyone should feel free to relay this guidance. You might have to drill down into the links there to see which models match up with which models that katrain is showing you, but the general point is that bigger nets are slower to run but stronger. Often stronger enough to compensate or more than compensate being slower, but depends on hardware.

https://github.com/lightvector/KataGo#other-questions

Thanks a lot, @lightvector!

_________________
“The only difference between me and a madman is that I’m not mad.” — Salvador Dali ★ Play a slooooow correspondence game with me on OGS?

Hi,
Just my two cents : after a bot is weakened, it would be a good thing to prevent it to ignore important ataris.

It feels really unnatural and disappointing when you play atari to a huge chain of stones and the bot just doesn't see it. Capture, game over.

It tries to do this by making 'important enough' moves non-optional, but it can be hard to determine which ones are important enough. However, a super hard to see 30 point life and death problem and a 15 stone atari can be essentially equivalent.
Also when this is a bit too highly tuned, it's easy to get an AI which is accidentally high dan.
If you save example games in which this happens and send them, it will help in creating something that might avoid this.

How about making weak KataGo not playing a move worse than what low level GnuGo would pick. That would probably be a reasonable way to discard wacky bad moves a 25 kyu wouldn't do.

How do you know a move is better or worse than a certain level of GnuGo?

The only thing you have is:

* The policy network values, as a probability map over all moves
* The score/win rate/lcb/utility for the current position for a limited number (~10) of moves.

Most weaker AIs just use the policy, which is around 5d by itself, but does on occasion make mistakes. Less so for the 20b network, but still.
The new KataHandicap setup allows to submit additional queries apart from the one for analysis, so it could be used to double-check the move chosen based on the policy (and then? play the top move instead?) but it requires some serious testing. It can easily end up as the '15k' AI that suddenly plays like an extremely murderous 9p when you invade a moyo.

Ask strong KataGo.

So my idea the workflow would be like:

1) Generate move suggestion with weak KataGo (from just policy or whatever else you do to cripple it), called wKn, where n is currently 0
2) Generate move suggestion with GnuGo, called g0
3) Ask strong KataGo (doesn't need to be that strong, maybe just 100 playouts is enough) is g0 > wKn in winrate or score? (doesn't matter really which one used)
3a) If n > 10 (or so) play wKn anyway
3b) If yes: get another choice from weak KataGo (either further down policy, or re-roll random number generator), called wKn where n is now n+1, go to step 3 and compare to this new move
3c) If no, play wKn as it's not a retarded move.

This would require packaging another executable for multiple platforms, GTP engine support, with all the UI elements for configuring it and potential errors that can occur.
Way too much work for something like this.

Hmm, I wonder if a query with wideRootNoise can be used for this, that should get back a lot of moveInfos for relatively few visits.

I am a new user of KaTrain! (I have fooled around a lot with katago)

Nice work!

The calibrated rank, what scale is it aiming for (EGF, AGA, IGS)?

Is it possible to let two bots play each other at different calibrated ranks?

Thanks!

When trying to get a calibrated rank, don't you think it could be useful to train a very small net (6 block) to just tell how probable a certain move is to be played by an human? And then just skip moves with a (very) low score.

(I bet an atari on a 25 stone group would get a high score)

It's calibrated against GnuGO and Pachi - I have no idea what that corresponds to, but the bots seem to be holding their OGS ranks within around 2k, except for katrain-18k which varies a bit more.
Self-play of the same AI is only possible outside the program, see e.g. selfplay in https://github.com/sanderland/katrain-bots

I am not sure why you think a 6b net policy is closer to human play than the 15b policy.

I am sorry if I was unclear. But the smaller net should not be trained to play go. It should be taught to tell how probable it is that a certain move would be played by an human. It should not be trained through self play. It should be train by "watching" human games of all levels. It must be trained on human games (that's why earlier versions of AlplhaGo could give that info) It's a totally new net trained from scratch. I suggested 6b so it would be as fast as possible. And it should not use playouts. It should be used i parallel with a net that plays go. It should help the weakening process to not skip moves that would be obvious to an human.

Did that made it any clearer?

It should easily be able to distinct between a 25 stone atari and a slightly more difficult life and death problem.

Most life and death problems are not as big as capturing 25 stones vs. letting them escape.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

I think you are missing the point!
This is not playing and it's not choosing between two different moves.
It is a mean to weaken the bot so it does not always find the correct move in a tricky tsumego.
But still be able to answer to a simple atari on a "big" group.
A strong bot will "always" know where to play.
When you weaken it you want it to not always play the best move.
But that will unfortunately make the bot not answering ataris.
Because not answering an atari and not live with a tricky tsumego are all the same to the bot.
But for an human not answering an atari looks plain stupid.
So we don't want the bot to skip answering an atari as to make it's mistakes more human like.

What I proposed was a mean I think will help to accomplish that.

Perhaps you are missing my point, I dunno.



It's a comparison between the size of two different moves, right?



My guess is that you mean "means". I.e., it is a way to weaken the bot, not an average.



To which the question is how big? I was pointing out that the size of capturing a 25 stone group is rather bigger than most tsumego problems or life and death problems than arise on the go board.

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.