Mike Novack wrote:
Krama wrote:
A neural network trained by using pro games + MCTS
This is how I imagine the future should look like, and with this we could probably get to insei level.
This bears discussion. First we should remember the history of the MCTS approach. While immediately obvious that major advance from the then strongest "go knowledge" based AI porgrams, the "infant" MCTS programs weren't stronger than the infant neural net programs. And MCTS may have "hit the wall" (because computationally intensive).
The current neural net programs are trained to attempt to predict THE move that an expert would make and play on that basis alone. With this training based on just the record of games between experts. There are a number of possibilities.
1) "add" training based on problems. I believe it is possible to have placements for the rest of the board (outside the problem area) such that ONLY correctly solving the problem brings victory. I put "add" in quotes because back and forth training (on different bases) could be used for "annealing" << if you don't know how neural nets are trained, won't make much sense >>
2) Returning the BEST single move (what percentage success with that) is not the same thing as returning a small set of moves and asking what the percentage success that this set contain the best move. Needs to be investigated, because if that set is small (for a very high percentage of success) MCTS could then be used as an evaluator to choose between them. Potential for significant time gain. Remember, there are TWO ways that a MCTS program can fail to find the best move. One is to incorrectly identify the best from the set that was the root of the trees. But the other is not having the best move in that set.
This is why a NN would give top 3 moves that it thinks are the best and then MCTS could play using those three moves and find out which should be the best.
Also what happens if you take top 3 moves from NN and then ask it to find top 3 moves for each of those moves (this time playing as opponent) and then repeat again. then out of them evaluate those 27 moves somehow and pick the best few and let it run in MCTS.
Also on your idea to train NN with L&D problems is something I have been thinking of and it shouldn't be hard.
Setup a L&D problem in one part of the board and setup the rest of the board with live groups that have eyes and make the score such that L&D problem needs to be solved in order to win the game.