yoyoma wrote:I saw somewhere (compgo?) that the Leela author said he is not running Leela on CGOS. So it is an unofficial test by someone else.
I think it's a test by pookpooi actually.
The most interesting would probably be to run Leela with 1 cpu and no GPU. Then we know how much the GPU is worth. I'm not sure it affects the conclusion vs Zenith because Zenith can't use the GPU anyway, but that is not Leela's fault.
Leela 0.9.0 defeats Zen again (this time without pondering, still using much more time than Zen)
Zenit Go 6 [setting: 20 seconds per move]
Leela 0.9.0 GPU [setting: 20 minutes for the game] (used more than 40 minutes)
CPU Intel 2500K GPU Nvidia GTX 970
Leela's advice towards Zen: Move 91 should have been at J10. Zen looses with an empty triangle (52% => 47% win)
Zen was still quite sure to win the game at this time (60% win)
This is a typical diagramm of an opening I play with pretty good success. After analyzing it for 20 minutes, Leela gives it only about 41.5% for White with the slow 'a' as best continuation for White. I am interested in what CS or Zen say to this position, and how they would continue for White. I usually continue with 'b' or 'c'.
Given that you are 5D, can't you just play this out vs Leela, and see what happens after a bunch of moves? Either it'll show you a refutation, or its score will go towards 50%.
Pippen wrote:This is a typical diagramm of an opening I play with pretty good success. After analyzing it for 20 minutes, Leela gives it only about 41.5% for White with the slow 'a' as best continuation for White. I am interested in what CS or Zen say to this position, and how they would continue for White. I usually continue with 'b' or 'c'.
Zen suggest k4 with 53% for white. But I'd rather trust Leela. If you look more closely at its Net Win %, it says w is winning by 28% (i.e. losing the game) which is, I believe, more accurate than Win %. At least that's what those Deep Mind guys were looking at when they were testing AlphaGo.
Zen suggest k4 with 53% for white. But I'd rather trust Leela. If you look more closely at its Net Win %, it says w is winning by 28% (i.e. losing the game) which is, I believe, more accurate than Win %. At least that's what those Deep Mind guys were looking at when they were testing AlphaGo.
Interesting. How can there be such a difference between the two probabilities?
I rechecked and here is what Leela says in detail: D2(best move): Win% 42, MC win% 51.7, Net win% 31.8, Net prob% 29. What is the meaning of all these numbers anyway? I always thought that win% matters, the rest not so much.
Pippen wrote:This is a typical diagramm of an opening I play with pretty good success. After analyzing it for 20 minutes, Leela gives it only about 41.5% for White with the slow 'a' as best continuation for White. I am interested in what CS or Zen say to this position, and how they would continue for White. I usually continue with 'b' or 'c'.
Pippen wrote:
I rechecked and here is what Leela says in detail: D2(best move): Win% 42, MC win% 51.7, Net win% 31.8, Net prob% 29. What is the meaning of all these numbers anyway? I always thought that win% matters, the rest not so much.
It's explained on Leela's webpage.
FAQ: "What do the columns in the Analysis Window mean?"
Simulations: the number of Monte Carlo playouts used to investigate the move. More simulations means more confidence in the winrate as the move has been investigated deeper.
Win%: this represents Leela's best guess as to how likely it is for the player to move to win the game.
MC Win%: the likelyhood that the player to move wins the game, as determined by randomized Monte Carlo playouts from the current position. This is a factor in the Win% calculation. Net Win%: the likelyhood that the player to move wins the game, as determined by analyzing the position with a neural network. This is a factor in the Win% calculation.
Net Prob%: the probability that a pro player would play this move, as estimated by the Neural Network.
PV: the principal variation. The sequence of suggested moves for both players that Leela believes is optimal.
How is Net win % and Net Prob win % calculated? What does Leela do (in simple words)? For example, in MC win%, Leela is just playing many random games and e.g. if Black wins 6000 out of 10.000 random games it will give Black 60% winrate.
Its a bit like human intuition. That net looked at millions of positions and who was winning in the end, and learned to develop an intuition saying "this looks like black is going to win" or "this looks like its still even". This intuition thinks black is likely to win from here(probably the net was trained with pro games or strong amateur games(KGS? I know Alphago started out with KGS games), maybe also with selfplay-games of Leela, I dont know.
(52% => 47% win)
I am interested in what CS or Zen say to this position, and how they would continue for White. I usually continue with 'b' or 'c'. 